Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=1),
 'dropout': 0.30000000000000004,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 300,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 300,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.628169
Epoch 1
Validation auc = 0.695305
Epoch 2
Validation auc = 0.695775
Epoch 3
Validation auc = 0.698592
Epoch 4
Loss = 1.0586e+00, PNorm = 40.6669, GNorm = 12.6522, lr_0 = 6.5000e-04
Validation auc = 0.730986
Epoch 5
Validation auc = 0.719718
Epoch 6
Validation auc = 0.658685
Epoch 7
Validation auc = 0.630986
Epoch 8
Validation auc = 0.614554
Epoch 9
Loss = 9.4697e-01, PNorm = 40.7117, GNorm = 11.4273, lr_0 = 9.9743e-04
Validation auc = 0.592488
Epoch 10
Validation auc = 0.669484
Epoch 11
Validation auc = 0.636150
Epoch 12
Validation auc = 0.654460
Epoch 13
Validation auc = 0.662441
Epoch 14
Loss = 8.9000e-01, PNorm = 40.7700, GNorm = 8.3154, lr_0 = 9.8890e-04
Validation auc = 0.714085
Epoch 15
Validation auc = 0.723474
Epoch 16
Validation auc = 0.698122
Epoch 17
Validation auc = 0.689671
Epoch 18
Validation auc = 0.692488
Epoch 19
Loss = 6.4386e-01, PNorm = 40.8093, GNorm = 5.7968, lr_0 = 9.8045e-04
Validation auc = 0.713146
Epoch 20
Validation auc = 0.737089
Epoch 21
Validation auc = 0.731925
Epoch 22
Validation auc = 0.715493
Epoch 23
Validation auc = 0.715962
Epoch 24
Loss = 7.5890e-01, PNorm = 40.8456, GNorm = 4.7717, lr_0 = 9.7207e-04
Validation auc = 0.718310
Epoch 25
Validation auc = 0.731925
Epoch 26
Validation auc = 0.747887
Epoch 27
Validation auc = 0.760094
Epoch 28
Validation auc = 0.762911
Epoch 29
Loss = 6.4495e-01, PNorm = 40.8848, GNorm = 5.2614, lr_0 = 9.6376e-04
Validation auc = 0.753052
Epoch 30
Validation auc = 0.729577
Epoch 31
Validation auc = 0.701408
Epoch 32
Validation auc = 0.689671
Epoch 33
Validation auc = 0.690610
Epoch 34
Loss = 6.2325e-01, PNorm = 40.9212, GNorm = 6.1661, lr_0 = 9.5552e-04
Validation auc = 0.687793
Epoch 35
Validation auc = 0.690141
Epoch 36
Validation auc = 0.678404
Epoch 37
Validation auc = 0.668545
Epoch 38
Validation auc = 0.644601
Epoch 39
Loss = 5.5793e-01, PNorm = 40.9536, GNorm = 3.7693, lr_0 = 9.4735e-04
Validation auc = 0.650235
Epoch 40
Validation auc = 0.648357
Epoch 41
Validation auc = 0.670423
Epoch 42
Validation auc = 0.678873
Epoch 43
Validation auc = 0.690141
Epoch 44
Loss = 5.1738e-01, PNorm = 40.9941, GNorm = 4.0626, lr_0 = 9.3925e-04
Validation auc = 0.697653
Epoch 45
Validation auc = 0.684038
Epoch 46
Validation auc = 0.674648
Epoch 47
Validation auc = 0.668545
Epoch 48
Validation auc = 0.652113
Epoch 49
Loss = 6.5326e-01, PNorm = 41.0340, GNorm = 5.2204, lr_0 = 9.3122e-04
Validation auc = 0.645540
Epoch 50
Validation auc = 0.654930
Epoch 51
Validation auc = 0.657277
Epoch 52
Validation auc = 0.657277
Epoch 53
Validation auc = 0.651643
Epoch 54
Loss = 5.4901e-01, PNorm = 41.0795, GNorm = 5.6308, lr_0 = 9.2326e-04
Validation auc = 0.646948
Epoch 55
Validation auc = 0.646948
Epoch 56
Validation auc = 0.648826
Epoch 57
Validation auc = 0.649296
Epoch 58
Validation auc = 0.641784
Epoch 59
Loss = 6.5911e-01, PNorm = 41.1254, GNorm = 5.2338, lr_0 = 9.1537e-04
Validation auc = 0.638967
Epoch 60
Validation auc = 0.642254
Epoch 61
Validation auc = 0.638028
Epoch 62
Validation auc = 0.641784
Epoch 63
Validation auc = 0.648357
Epoch 64
Loss = 5.2210e-01, PNorm = 41.1934, GNorm = 4.7278, lr_0 = 9.0754e-04
Validation auc = 0.646479
Epoch 65
Validation auc = 0.641784
Epoch 66
Validation auc = 0.622535
Epoch 67
Validation auc = 0.616901
Epoch 68
Validation auc = 0.618779
Epoch 69
Loss = 4.8703e-01, PNorm = 41.2472, GNorm = 3.2865, lr_0 = 8.9978e-04
Validation auc = 0.628638
Epoch 70
Validation auc = 0.652582
Epoch 71
Validation auc = 0.642723
Epoch 72
Validation auc = 0.647887
Epoch 73
Validation auc = 0.642723
Epoch 74
Loss = 6.0897e-01, PNorm = 41.3447, GNorm = 4.1058, lr_0 = 8.9209e-04
Validation auc = 0.634272
Epoch 75
Validation auc = 0.629108
Epoch 76
Validation auc = 0.628169
Epoch 77
Validation auc = 0.620188
Epoch 78
Validation auc = 0.615962
Epoch 79
Loss = 6.5077e-01, PNorm = 41.4027, GNorm = 4.8542, lr_0 = 8.8447e-04
Validation auc = 0.621596
Epoch 80
Validation auc = 0.618779
Epoch 81
Validation auc = 0.621596
Epoch 82
Validation auc = 0.629108
Epoch 83
Validation auc = 0.631925
Epoch 84
Loss = 4.4870e-01, PNorm = 41.4659, GNorm = 2.9592, lr_0 = 8.7691e-04
Validation auc = 0.630516
Epoch 85
Validation auc = 0.633803
Epoch 86
Validation auc = 0.636150
Epoch 87
Validation auc = 0.635681
Epoch 88
Validation auc = 0.646479
Epoch 89
Loss = 5.3074e-01, PNorm = 41.5356, GNorm = 4.7615, lr_0 = 8.6941e-04
Validation auc = 0.656338
Epoch 90
Validation auc = 0.653052
Epoch 91
Validation auc = 0.650704
Epoch 92
Validation auc = 0.653521
Epoch 93
Validation auc = 0.657277
Epoch 94
Loss = 5.0271e-01, PNorm = 41.5936, GNorm = 2.9955, lr_0 = 8.6198e-04
Validation auc = 0.681221
Epoch 95
Validation auc = 0.676526
Epoch 96
Validation auc = 0.652582
Epoch 97
Validation auc = 0.641315
Epoch 98
Validation auc = 0.642723
Epoch 99
Loss = 4.0967e-01, PNorm = 41.6846, GNorm = 2.3018, lr_0 = 8.5461e-04
Validation auc = 0.651174
Epoch 100
Validation auc = 0.656808
Epoch 101
Validation auc = 0.656338
Epoch 102
Validation auc = 0.665258
Epoch 103
Validation auc = 0.676526
Epoch 104
Loss = 5.1816e-01, PNorm = 41.7601, GNorm = 4.4473, lr_0 = 8.4730e-04
Validation auc = 0.679812
Epoch 105
Validation auc = 0.683099
Epoch 106
Validation auc = 0.672300
Epoch 107
Validation auc = 0.668545
Epoch 108
Validation auc = 0.668075
Epoch 109
Loss = 5.3604e-01, PNorm = 41.8502, GNorm = 2.7722, lr_0 = 8.4006e-04
Validation auc = 0.669484
Epoch 110
Validation auc = 0.665258
Epoch 111
Validation auc = 0.668075
Epoch 112
Validation auc = 0.667606
Epoch 113
Validation auc = 0.666197
Epoch 114
Loss = 4.8188e-01, PNorm = 41.9245, GNorm = 2.7175, lr_0 = 8.3288e-04
Validation auc = 0.670892
Epoch 115
Validation auc = 0.668545
Epoch 116
Validation auc = 0.669484
Epoch 117
Validation auc = 0.670892
Epoch 118
Validation auc = 0.675117
Epoch 119
Loss = 4.4285e-01, PNorm = 42.0460, GNorm = 4.2058, lr_0 = 8.2576e-04
Validation auc = 0.684977
Epoch 120
Validation auc = 0.700000
Epoch 121
Validation auc = 0.707042
Epoch 122
Validation auc = 0.706573
Epoch 123
Validation auc = 0.700939
Epoch 124
Loss = 4.2226e-01, PNorm = 42.1521, GNorm = 3.8331, lr_0 = 8.1870e-04
Validation auc = 0.684977
Epoch 125
Validation auc = 0.676056
Epoch 126
Validation auc = 0.674648
Epoch 127
Validation auc = 0.678873
Epoch 128
Validation auc = 0.672770
Epoch 129
Loss = 3.5606e-01, PNorm = 42.2504, GNorm = 1.9712, lr_0 = 8.1170e-04
Validation auc = 0.659155
Epoch 130
Validation auc = 0.666197
Epoch 131
Validation auc = 0.673709
Epoch 132
Validation auc = 0.673709
Epoch 133
Validation auc = 0.666197
Epoch 134
Loss = 4.2385e-01, PNorm = 42.3388, GNorm = 3.3083, lr_0 = 8.0476e-04
Validation auc = 0.653521
Epoch 135
Validation auc = 0.656338
Epoch 136
Validation auc = 0.672770
Epoch 137
Validation auc = 0.689202
Epoch 138
Validation auc = 0.688263
Epoch 139
Loss = 3.8030e-01, PNorm = 42.4286, GNorm = 2.9730, lr_0 = 7.9788e-04
Validation auc = 0.687324
Epoch 140
Validation auc = 0.697653
Epoch 141
Validation auc = 0.707512
Epoch 142
Validation auc = 0.718310
Epoch 143
Validation auc = 0.723944
Epoch 144
Loss = 4.4222e-01, PNorm = 42.5211, GNorm = 2.4797, lr_0 = 7.9106e-04
Validation auc = 0.722535
Epoch 145
Validation auc = 0.716901
Epoch 146
Validation auc = 0.715023
Epoch 147
Validation auc = 0.717371
Epoch 148
Validation auc = 0.716901
Epoch 149
Loss = 3.7644e-01, PNorm = 42.6240, GNorm = 3.6333, lr_0 = 7.8430e-04
Validation auc = 0.711268
Epoch 150
Validation auc = 0.717371
Epoch 151
Validation auc = 0.723474
Epoch 152
Validation auc = 0.728169
Epoch 153
Validation auc = 0.736150
Epoch 154
Loss = 4.3723e-01, PNorm = 42.7339, GNorm = 2.5887, lr_0 = 7.7759e-04
Validation auc = 0.749765
Epoch 155
Validation auc = 0.746948
Epoch 156
Validation auc = 0.740376
Epoch 157
Validation auc = 0.736150
Epoch 158
Validation auc = 0.726761
Epoch 159
Loss = 3.6100e-01, PNorm = 42.8272, GNorm = 3.5246, lr_0 = 7.7095e-04
Validation auc = 0.722066
Epoch 160
Validation auc = 0.715023
Epoch 161
Validation auc = 0.711737
Epoch 162
Validation auc = 0.722535
Epoch 163
Validation auc = 0.727230
Epoch 164
Loss = 4.1562e-01, PNorm = 42.9251, GNorm = 2.8812, lr_0 = 7.6436e-04
Validation auc = 0.731455
Epoch 165
Validation auc = 0.735681
Epoch 166
Validation auc = 0.729577
Epoch 167
Validation auc = 0.715493
Epoch 168
Validation auc = 0.701878
Epoch 169
Loss = 3.3769e-01, PNorm = 43.0134, GNorm = 3.8247, lr_0 = 7.5782e-04
Validation auc = 0.699531
Epoch 170
Validation auc = 0.697653
Epoch 171
Validation auc = 0.703286
Epoch 172
Validation auc = 0.700939
Epoch 173
Validation auc = 0.704225
Epoch 174
Loss = 3.7853e-01, PNorm = 43.1116, GNorm = 2.9030, lr_0 = 7.5134e-04
Validation auc = 0.715023
Epoch 175
Validation auc = 0.716901
Epoch 176
Validation auc = 0.728638
Epoch 177
Validation auc = 0.728638
Epoch 178
Validation auc = 0.727700
Epoch 179
Loss = 3.6827e-01, PNorm = 43.2310, GNorm = 2.7030, lr_0 = 7.4492e-04
Validation auc = 0.729577
Epoch 180
Validation auc = 0.729577
Epoch 181
Validation auc = 0.710798
Epoch 182
Validation auc = 0.704695
Epoch 183
Validation auc = 0.704225
Epoch 184
Loss = 3.2601e-01, PNorm = 43.3343, GNorm = 2.4061, lr_0 = 7.3855e-04
Validation auc = 0.709859
Epoch 185
Validation auc = 0.715023
Epoch 186
Validation auc = 0.715493
Epoch 187
Validation auc = 0.711268
Epoch 188
Validation auc = 0.712207
Epoch 189
Loss = 4.9626e-01, PNorm = 43.4414, GNorm = 4.7753, lr_0 = 7.3224e-04
Validation auc = 0.719249
Epoch 190
Validation auc = 0.730516
Epoch 191
Validation auc = 0.730986
Epoch 192
Validation auc = 0.726761
Epoch 193
Validation auc = 0.723944
Epoch 194
Loss = 3.4785e-01, PNorm = 43.5355, GNorm = 2.0181, lr_0 = 7.2598e-04
Validation auc = 0.723005
Epoch 195
Validation auc = 0.734272
Epoch 196
Validation auc = 0.734272
Epoch 197
Validation auc = 0.731455
Epoch 198
Validation auc = 0.733333
Epoch 199
Loss = 2.9465e-01, PNorm = 43.6278, GNorm = 2.5281, lr_0 = 7.1977e-04
Validation auc = 0.729577
Epoch 200
Validation auc = 0.723474
Epoch 201
Validation auc = 0.728638
Epoch 202
Validation auc = 0.730516
Epoch 203
Validation auc = 0.730047
Epoch 204
Loss = 3.0448e-01, PNorm = 43.7115, GNorm = 2.9157, lr_0 = 7.1362e-04
Validation auc = 0.728169
Epoch 205
Validation auc = 0.727700
Epoch 206
Validation auc = 0.729108
Epoch 207
Validation auc = 0.730047
Epoch 208
Validation auc = 0.735681
Epoch 209
Loss = 2.1663e-01, PNorm = 43.8098, GNorm = 2.9022, lr_0 = 7.0752e-04
Validation auc = 0.734742
Epoch 210
Validation auc = 0.734272
Epoch 211
Validation auc = 0.730516
Epoch 212
Validation auc = 0.730516
Epoch 213
Validation auc = 0.730047
Epoch 214
Loss = 3.1711e-01, PNorm = 43.9030, GNorm = 3.3697, lr_0 = 7.0147e-04
Validation auc = 0.729108
Epoch 215
Validation auc = 0.727230
Epoch 216
Validation auc = 0.720188
Epoch 217
Validation auc = 0.715493
Epoch 218
Validation auc = 0.715962
Epoch 219
Loss = 2.7173e-01, PNorm = 43.9853, GNorm = 2.8156, lr_0 = 6.9548e-04
Validation auc = 0.718310
Epoch 220
Validation auc = 0.717840
Epoch 221
Validation auc = 0.720657
Epoch 222
Validation auc = 0.721596
Epoch 223
Validation auc = 0.727230
Epoch 224
Loss = 2.6804e-01, PNorm = 44.0681, GNorm = 3.0562, lr_0 = 6.8953e-04
Validation auc = 0.730047
Epoch 225
Validation auc = 0.736150
Epoch 226
Validation auc = 0.735211
Epoch 227
Validation auc = 0.731455
Epoch 228
Validation auc = 0.723474
Epoch 229
Loss = 3.6117e-01, PNorm = 44.1421, GNorm = 2.2142, lr_0 = 6.8364e-04
Validation auc = 0.730986
Epoch 230
Validation auc = 0.729108
Epoch 231
Validation auc = 0.727230
Epoch 232
Validation auc = 0.725352
Epoch 233
Validation auc = 0.724883
Epoch 234
Loss = 2.2079e-01, PNorm = 44.2325, GNorm = 1.7093, lr_0 = 6.7779e-04
Validation auc = 0.723944
Epoch 235
Validation auc = 0.725822
Epoch 236
Validation auc = 0.723944
Epoch 237
Validation auc = 0.728169
Epoch 238
Validation auc = 0.729108
Epoch 239
Loss = 2.8374e-01, PNorm = 44.3233, GNorm = 1.9163, lr_0 = 6.7200e-04
Validation auc = 0.728638
Epoch 240
Validation auc = 0.733333
Epoch 241
Validation auc = 0.736620
Epoch 242
Validation auc = 0.743662
Epoch 243
Validation auc = 0.740376
Epoch 244
Loss = 2.6964e-01, PNorm = 44.4131, GNorm = 1.9107, lr_0 = 6.6625e-04
Validation auc = 0.740376
Epoch 245
Validation auc = 0.746479
Epoch 246
Validation auc = 0.750704
Epoch 247
Validation auc = 0.750235
Epoch 248
Validation auc = 0.746948
Epoch 249
Loss = 3.0184e-01, PNorm = 44.4966, GNorm = 2.6694, lr_0 = 6.6056e-04
Validation auc = 0.739906
Epoch 250
Validation auc = 0.735681
Epoch 251
Validation auc = 0.733333
Epoch 252
Validation auc = 0.731455
Epoch 253
Validation auc = 0.730516
Epoch 254
Loss = 2.6537e-01, PNorm = 44.5765, GNorm = 2.5545, lr_0 = 6.5491e-04
Validation auc = 0.730047
Epoch 255
Validation auc = 0.731455
Epoch 256
Validation auc = 0.732394
Epoch 257
Validation auc = 0.733803
Epoch 258
Validation auc = 0.734272
Epoch 259
Loss = 2.5517e-01, PNorm = 44.6631, GNorm = 1.3606, lr_0 = 6.4931e-04
Validation auc = 0.735681
Epoch 260
Validation auc = 0.733333
Epoch 261
Validation auc = 0.729108
Epoch 262
Validation auc = 0.723474
Epoch 263
Validation auc = 0.717840
Epoch 264
Loss = 3.0369e-01, PNorm = 44.7385, GNorm = 2.0090, lr_0 = 6.4376e-04
Validation auc = 0.715962
Epoch 265
Validation auc = 0.714085
Epoch 266
Validation auc = 0.713615
Epoch 267
Validation auc = 0.715493
Epoch 268
Validation auc = 0.717840
Epoch 269
Loss = 2.3530e-01, PNorm = 44.8324, GNorm = 2.5031, lr_0 = 6.3826e-04
Validation auc = 0.717840
Epoch 270
Validation auc = 0.721596
Epoch 271
Validation auc = 0.722535
Epoch 272
Validation auc = 0.724413
Epoch 273
Validation auc = 0.726761
Epoch 274
Loss = 1.9287e-01, PNorm = 44.9034, GNorm = 1.1133, lr_0 = 6.3280e-04
Validation auc = 0.728638
Epoch 275
Validation auc = 0.726761
Epoch 276
Validation auc = 0.722535
Epoch 277
Validation auc = 0.720188
Epoch 278
Validation auc = 0.723944
Epoch 279
Loss = 2.6623e-01, PNorm = 44.9745, GNorm = 3.6322, lr_0 = 6.2739e-04
Validation auc = 0.723005
Epoch 280
Validation auc = 0.719249
Epoch 281
Validation auc = 0.715962
Epoch 282
Validation auc = 0.713146
Epoch 283
Validation auc = 0.715962
Epoch 284
Loss = 2.0346e-01, PNorm = 45.0538, GNorm = 3.4675, lr_0 = 6.2203e-04
Validation auc = 0.715493
Epoch 285
Validation auc = 0.723005
Epoch 286
Validation auc = 0.728169
Epoch 287
Validation auc = 0.721127
Epoch 288
Validation auc = 0.717371
Epoch 289
Loss = 2.7355e-01, PNorm = 45.1216, GNorm = 3.2732, lr_0 = 6.1671e-04
Validation auc = 0.713146
Epoch 290
Validation auc = 0.714085
Epoch 291
Validation auc = 0.707042
Epoch 292
Validation auc = 0.712207
Epoch 293
Validation auc = 0.710329
Epoch 294
Loss = 2.7110e-01, PNorm = 45.2007, GNorm = 4.1903, lr_0 = 6.1144e-04
Validation auc = 0.708451
Epoch 295
Validation auc = 0.709859
Epoch 296
Validation auc = 0.720657
Epoch 297
Validation auc = 0.723474
Epoch 298
Validation auc = 0.725352
Epoch 299
Loss = 2.2368e-01, PNorm = 45.3015, GNorm = 1.8506, lr_0 = 6.0621e-04
Validation auc = 0.722535
Model 0 best validation auc = 0.762911 on epoch 28
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.856111
Ensemble test auc = 0.856111
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.649765
Epoch 1
Validation auc = 0.674178
Epoch 2
Validation auc = 0.667606
Epoch 3
Validation auc = 0.668545
Epoch 4
Loss = 1.0876e+00, PNorm = 40.6664, GNorm = 15.9575, lr_0 = 6.5000e-04
Validation auc = 0.664789
Epoch 5
Validation auc = 0.671362
Epoch 6
Validation auc = 0.660563
Epoch 7
Validation auc = 0.659155
Epoch 8
Validation auc = 0.669484
Epoch 9
Loss = 1.0050e+00, PNorm = 40.7127, GNorm = 9.4112, lr_0 = 9.9743e-04
Validation auc = 0.670423
Epoch 10
Validation auc = 0.734742
Epoch 11
Validation auc = 0.642723
Epoch 12
Validation auc = 0.659624
Epoch 13
Validation auc = 0.672300
Epoch 14
Loss = 7.5446e-01, PNorm = 40.7584, GNorm = 7.9101, lr_0 = 9.8890e-04
Validation auc = 0.732394
Epoch 15
Validation auc = 0.777934
Epoch 16
Validation auc = 0.690610
Epoch 17
Validation auc = 0.690610
Epoch 18
Validation auc = 0.705634
Epoch 19
Loss = 6.8242e-01, PNorm = 40.7987, GNorm = 6.0677, lr_0 = 9.8045e-04
Validation auc = 0.683568
Epoch 20
Validation auc = 0.663380
Epoch 21
Validation auc = 0.658685
Epoch 22
Validation auc = 0.661972
Epoch 23
Validation auc = 0.666197
Epoch 24
Loss = 6.5581e-01, PNorm = 40.8466, GNorm = 4.5116, lr_0 = 9.7207e-04
Validation auc = 0.665258
Epoch 25
Validation auc = 0.664789
Epoch 26
Validation auc = 0.668545
Epoch 27
Validation auc = 0.682160
Epoch 28
Validation auc = 0.699061
Epoch 29
Loss = 5.6465e-01, PNorm = 40.8680, GNorm = 4.2024, lr_0 = 9.6376e-04
Validation auc = 0.712676
Epoch 30
Validation auc = 0.706573
Epoch 31
Validation auc = 0.697653
Epoch 32
Validation auc = 0.686385
Epoch 33
Validation auc = 0.666667
Epoch 34
Loss = 5.9160e-01, PNorm = 40.9107, GNorm = 9.3277, lr_0 = 9.5552e-04
Validation auc = 0.640845
Epoch 35
Validation auc = 0.631455
Epoch 36
Validation auc = 0.620657
Epoch 37
Validation auc = 0.632394
Epoch 38
Validation auc = 0.625822
Epoch 39
Loss = 6.4413e-01, PNorm = 40.9579, GNorm = 4.8620, lr_0 = 9.4735e-04
Validation auc = 0.621127
Epoch 40
Validation auc = 0.617371
Epoch 41
Validation auc = 0.618310
Epoch 42
Validation auc = 0.614554
Epoch 43
Validation auc = 0.613146
Epoch 44
Loss = 5.8825e-01, PNorm = 40.9844, GNorm = 3.8137, lr_0 = 9.3925e-04
Validation auc = 0.614085
Epoch 45
Validation auc = 0.631455
Epoch 46
Validation auc = 0.635211
Epoch 47
Validation auc = 0.625822
Epoch 48
Validation auc = 0.632864
Epoch 49
Loss = 5.8276e-01, PNorm = 41.0387, GNorm = 4.7174, lr_0 = 9.3122e-04
Validation auc = 0.635681
Epoch 50
Validation auc = 0.620657
Epoch 51
Validation auc = 0.626761
Epoch 52
Validation auc = 0.658685
Epoch 53
Validation auc = 0.652582
Epoch 54
Loss = 5.7388e-01, PNorm = 41.0799, GNorm = 2.5862, lr_0 = 9.2326e-04
Validation auc = 0.643192
Epoch 55
Validation auc = 0.643192
Epoch 56
Validation auc = 0.639437
Epoch 57
Validation auc = 0.649765
Epoch 58
Validation auc = 0.672770
Epoch 59
Loss = 5.8007e-01, PNorm = 41.1300, GNorm = 5.0977, lr_0 = 9.1537e-04
Validation auc = 0.650704
Epoch 60
Validation auc = 0.637559
Epoch 61
Validation auc = 0.661502
Epoch 62
Validation auc = 0.662911
Epoch 63
Validation auc = 0.657746
Epoch 64
Loss = 5.7533e-01, PNorm = 41.1737, GNorm = 4.6470, lr_0 = 9.0754e-04
Validation auc = 0.653052
Epoch 65
Validation auc = 0.655399
Epoch 66
Validation auc = 0.668075
Epoch 67
Validation auc = 0.665258
Epoch 68
Validation auc = 0.645540
Epoch 69
Loss = 5.4912e-01, PNorm = 41.2251, GNorm = 3.4596, lr_0 = 8.9978e-04
Validation auc = 0.631925
Epoch 70
Validation auc = 0.637089
Epoch 71
Validation auc = 0.642723
Epoch 72
Validation auc = 0.640376
Epoch 73
Validation auc = 0.643192
Epoch 74
Loss = 5.6889e-01, PNorm = 41.2884, GNorm = 3.3854, lr_0 = 8.9209e-04
Validation auc = 0.638028
Epoch 75
Validation auc = 0.638967
Epoch 76
Validation auc = 0.634742
Epoch 77
Validation auc = 0.636150
Epoch 78
Validation auc = 0.645070
Epoch 79
Loss = 5.4145e-01, PNorm = 41.3278, GNorm = 2.9870, lr_0 = 8.8447e-04
Validation auc = 0.645070
Epoch 80
Validation auc = 0.652582
Epoch 81
Validation auc = 0.648826
Epoch 82
Validation auc = 0.640376
Epoch 83
Validation auc = 0.635211
Epoch 84
Loss = 4.8474e-01, PNorm = 41.3881, GNorm = 3.2753, lr_0 = 8.7691e-04
Validation auc = 0.644131
Epoch 85
Validation auc = 0.643662
Epoch 86
Validation auc = 0.645540
Epoch 87
Validation auc = 0.653991
Epoch 88
Validation auc = 0.647887
Epoch 89
Loss = 4.7768e-01, PNorm = 41.4428, GNorm = 2.8218, lr_0 = 8.6941e-04
Validation auc = 0.647418
Epoch 90
Validation auc = 0.646948
Epoch 91
Validation auc = 0.664789
Epoch 92
Validation auc = 0.700939
Epoch 93
Validation auc = 0.701878
Epoch 94
Loss = 5.0753e-01, PNorm = 41.5239, GNorm = 2.3757, lr_0 = 8.6198e-04
Validation auc = 0.696244
Epoch 95
Validation auc = 0.680751
Epoch 96
Validation auc = 0.669484
Epoch 97
Validation auc = 0.661972
Epoch 98
Validation auc = 0.671362
Epoch 99
Loss = 4.0298e-01, PNorm = 41.6022, GNorm = 2.5264, lr_0 = 8.5461e-04
Validation auc = 0.672770
Epoch 100
Validation auc = 0.675117
Epoch 101
Validation auc = 0.675117
Epoch 102
Validation auc = 0.666667
Epoch 103
Validation auc = 0.662441
Epoch 104
Loss = 5.3391e-01, PNorm = 41.6816, GNorm = 3.8458, lr_0 = 8.4730e-04
Validation auc = 0.659624
Epoch 105
Validation auc = 0.661502
Epoch 106
Validation auc = 0.662911
Epoch 107
Validation auc = 0.670892
Epoch 108
Validation auc = 0.684038
Epoch 109
Loss = 4.4263e-01, PNorm = 41.7511, GNorm = 2.5768, lr_0 = 8.4006e-04
Validation auc = 0.695305
Epoch 110
Validation auc = 0.704695
Epoch 111
Validation auc = 0.707512
Epoch 112
Validation auc = 0.708451
Epoch 113
Validation auc = 0.706573
Epoch 114
Loss = 4.4434e-01, PNorm = 41.8346, GNorm = 2.3888, lr_0 = 8.3288e-04
Validation auc = 0.700939
Epoch 115
Validation auc = 0.692488
Epoch 116
Validation auc = 0.686854
Epoch 117
Validation auc = 0.686854
Epoch 118
Validation auc = 0.696714
Epoch 119
Loss = 4.9984e-01, PNorm = 41.9289, GNorm = 2.5198, lr_0 = 8.2576e-04
Validation auc = 0.712676
Epoch 120
Validation auc = 0.728169
Epoch 121
Validation auc = 0.732394
Epoch 122
Validation auc = 0.727700
Epoch 123
Validation auc = 0.721596
Epoch 124
Loss = 4.4344e-01, PNorm = 42.0328, GNorm = 2.4286, lr_0 = 8.1870e-04
Validation auc = 0.711737
Epoch 125
Validation auc = 0.712207
Epoch 126
Validation auc = 0.710798
Epoch 127
Validation auc = 0.698122
Epoch 128
Validation auc = 0.702817
Epoch 129
Loss = 5.1110e-01, PNorm = 42.1331, GNorm = 3.1030, lr_0 = 8.1170e-04
Validation auc = 0.705164
Epoch 130
Validation auc = 0.704225
Epoch 131
Validation auc = 0.712207
Epoch 132
Validation auc = 0.719249
Epoch 133
Validation auc = 0.717371
Epoch 134
Loss = 4.0428e-01, PNorm = 42.2440, GNorm = 2.0037, lr_0 = 8.0476e-04
Validation auc = 0.722066
Epoch 135
Validation auc = 0.720188
Epoch 136
Validation auc = 0.707981
Epoch 137
Validation auc = 0.702347
Epoch 138
Validation auc = 0.699531
Epoch 139
Loss = 3.4503e-01, PNorm = 42.3450, GNorm = 2.4817, lr_0 = 7.9788e-04
Validation auc = 0.700939
Epoch 140
Validation auc = 0.709390
Epoch 141
Validation auc = 0.715493
Epoch 142
Validation auc = 0.716432
Epoch 143
Validation auc = 0.721127
Epoch 144
Loss = 3.9864e-01, PNorm = 42.4499, GNorm = 2.7479, lr_0 = 7.9106e-04
Validation auc = 0.723474
Epoch 145
Validation auc = 0.729577
Epoch 146
Validation auc = 0.737089
Epoch 147
Validation auc = 0.746479
Epoch 148
Validation auc = 0.747418
Epoch 149
Loss = 4.3380e-01, PNorm = 42.5534, GNorm = 3.7717, lr_0 = 7.8430e-04
Validation auc = 0.738028
Epoch 150
Validation auc = 0.744601
Epoch 151
Validation auc = 0.737559
Epoch 152
Validation auc = 0.726761
Epoch 153
Validation auc = 0.733333
Epoch 154
Loss = 3.4926e-01, PNorm = 42.6534, GNorm = 2.7665, lr_0 = 7.7759e-04
Validation auc = 0.732394
Epoch 155
Validation auc = 0.748357
Epoch 156
Validation auc = 0.756338
Epoch 157
Validation auc = 0.756808
Epoch 158
Validation auc = 0.751643
Epoch 159
Loss = 3.6341e-01, PNorm = 42.7375, GNorm = 2.2139, lr_0 = 7.7095e-04
Validation auc = 0.746479
Epoch 160
Validation auc = 0.755869
Epoch 161
Validation auc = 0.760563
Epoch 162
Validation auc = 0.755399
Epoch 163
Validation auc = 0.758685
Epoch 164
Loss = 3.8049e-01, PNorm = 42.8171, GNorm = 2.4252, lr_0 = 7.6436e-04
Validation auc = 0.755399
Epoch 165
Validation auc = 0.749765
Epoch 166
Validation auc = 0.748826
Epoch 167
Validation auc = 0.746009
Epoch 168
Validation auc = 0.724883
Epoch 169
Loss = 3.8821e-01, PNorm = 42.9019, GNorm = 1.6559, lr_0 = 7.5782e-04
Validation auc = 0.715493
Epoch 170
Validation auc = 0.717371
Epoch 171
Validation auc = 0.715023
Epoch 172
Validation auc = 0.717371
Epoch 173
Validation auc = 0.720657
Epoch 174
Loss = 3.8514e-01, PNorm = 43.0044, GNorm = 2.9102, lr_0 = 7.5134e-04
Validation auc = 0.724883
Epoch 175
Validation auc = 0.730047
Epoch 176
Validation auc = 0.730986
Epoch 177
Validation auc = 0.729577
Epoch 178
Validation auc = 0.734272
Epoch 179
Loss = 3.1922e-01, PNorm = 43.0983, GNorm = 2.4689, lr_0 = 7.4492e-04
Validation auc = 0.730986
Epoch 180
Validation auc = 0.730047
Epoch 181
Validation auc = 0.721596
Epoch 182
Validation auc = 0.735681
Epoch 183
Validation auc = 0.750235
Epoch 184
Loss = 3.7262e-01, PNorm = 43.1781, GNorm = 2.8077, lr_0 = 7.3855e-04
Validation auc = 0.761033
Epoch 185
Validation auc = 0.761033
Epoch 186
Validation auc = 0.758685
Epoch 187
Validation auc = 0.756338
Epoch 188
Validation auc = 0.750704
Epoch 189
Loss = 5.2751e-01, PNorm = 43.2531, GNorm = 4.5642, lr_0 = 7.3224e-04
Validation auc = 0.749296
Epoch 190
Validation auc = 0.746479
Epoch 191
Validation auc = 0.749296
Epoch 192
Validation auc = 0.744601
Epoch 193
Validation auc = 0.733803
Epoch 194
Loss = 3.9580e-01, PNorm = 43.3283, GNorm = 3.3242, lr_0 = 7.2598e-04
Validation auc = 0.738028
Epoch 195
Validation auc = 0.744131
Epoch 196
Validation auc = 0.753521
Epoch 197
Validation auc = 0.756808
Epoch 198
Validation auc = 0.756808
Epoch 199
Loss = 3.0891e-01, PNorm = 43.4174, GNorm = 1.9664, lr_0 = 7.1977e-04
Validation auc = 0.760094
Epoch 200
Validation auc = 0.755869
Epoch 201
Validation auc = 0.753052
Epoch 202
Validation auc = 0.753991
Epoch 203
Validation auc = 0.750704
Epoch 204
Loss = 2.8270e-01, PNorm = 43.5025, GNorm = 2.6705, lr_0 = 7.1362e-04
Validation auc = 0.747418
Epoch 205
Validation auc = 0.741315
Epoch 206
Validation auc = 0.751174
Epoch 207
Validation auc = 0.744131
Epoch 208
Validation auc = 0.739906
Epoch 209
Loss = 3.6347e-01, PNorm = 43.5936, GNorm = 3.9375, lr_0 = 7.0752e-04
Validation auc = 0.738028
Epoch 210
Validation auc = 0.733333
Epoch 211
Validation auc = 0.735681
Epoch 212
Validation auc = 0.742723
Epoch 213
Validation auc = 0.750704
Epoch 214
Loss = 2.4105e-01, PNorm = 43.6841, GNorm = 1.9146, lr_0 = 7.0147e-04
Validation auc = 0.756338
Epoch 215
Validation auc = 0.760563
Epoch 216
Validation auc = 0.763380
Epoch 217
Validation auc = 0.761033
Epoch 218
Validation auc = 0.754460
Epoch 219
Loss = 3.0605e-01, PNorm = 43.7635, GNorm = 2.8301, lr_0 = 6.9548e-04
Validation auc = 0.753052
Epoch 220
Validation auc = 0.754460
Epoch 221
Validation auc = 0.746479
Epoch 222
Validation auc = 0.746479
Epoch 223
Validation auc = 0.740845
Epoch 224
Loss = 3.8233e-01, PNorm = 43.8460, GNorm = 3.8259, lr_0 = 6.8953e-04
Validation auc = 0.733803
Epoch 225
Validation auc = 0.718779
Epoch 226
Validation auc = 0.718310
Epoch 227
Validation auc = 0.712676
Epoch 228
Validation auc = 0.713146
Epoch 229
Loss = 2.6902e-01, PNorm = 43.9145, GNorm = 2.7136, lr_0 = 6.8364e-04
Validation auc = 0.713615
Epoch 230
Validation auc = 0.711737
Epoch 231
Validation auc = 0.718779
Epoch 232
Validation auc = 0.728638
Epoch 233
Validation auc = 0.737559
Epoch 234
Loss = 2.9793e-01, PNorm = 44.0015, GNorm = 2.6566, lr_0 = 6.7779e-04
Validation auc = 0.741784
Epoch 235
Validation auc = 0.741784
Epoch 236
Validation auc = 0.742723
Epoch 237
Validation auc = 0.740845
Epoch 238
Validation auc = 0.748826
Epoch 239
Loss = 2.9457e-01, PNorm = 44.0980, GNorm = 2.9897, lr_0 = 6.7200e-04
Validation auc = 0.755399
Epoch 240
Validation auc = 0.752582
Epoch 241
Validation auc = 0.738498
Epoch 242
Validation auc = 0.732864
Epoch 243
Validation auc = 0.728638
Epoch 244
Loss = 3.8543e-01, PNorm = 44.1730, GNorm = 4.5694, lr_0 = 6.6625e-04
Validation auc = 0.726761
Epoch 245
Validation auc = 0.733333
Epoch 246
Validation auc = 0.733333
Epoch 247
Validation auc = 0.738967
Epoch 248
Validation auc = 0.743662
Epoch 249
Loss = 3.9622e-01, PNorm = 44.2576, GNorm = 7.8824, lr_0 = 6.6056e-04
Validation auc = 0.742723
Epoch 250
Validation auc = 0.731455
Epoch 251
Validation auc = 0.715023
Epoch 252
Validation auc = 0.709859
Epoch 253
Validation auc = 0.714085
Epoch 254
Loss = 3.5730e-01, PNorm = 44.3315, GNorm = 2.6143, lr_0 = 6.5491e-04
Validation auc = 0.714085
Epoch 255
Validation auc = 0.715493
Epoch 256
Validation auc = 0.716432
Epoch 257
Validation auc = 0.721596
Epoch 258
Validation auc = 0.720188
Epoch 259
Loss = 3.4132e-01, PNorm = 44.3949, GNorm = 5.0619, lr_0 = 6.4931e-04
Validation auc = 0.722066
Epoch 260
Validation auc = 0.730047
Epoch 261
Validation auc = 0.730047
Epoch 262
Validation auc = 0.732394
Epoch 263
Validation auc = 0.729108
Epoch 264
Loss = 3.3519e-01, PNorm = 44.4651, GNorm = 4.9413, lr_0 = 6.4376e-04
Validation auc = 0.730986
Epoch 265
Validation auc = 0.733333
Epoch 266
Validation auc = 0.729108
Epoch 267
Validation auc = 0.729577
Epoch 268
Validation auc = 0.725822
Epoch 269
Loss = 2.8320e-01, PNorm = 44.5310, GNorm = 1.9137, lr_0 = 6.3826e-04
Validation auc = 0.730047
Epoch 270
Validation auc = 0.729577
Epoch 271
Validation auc = 0.727230
Epoch 272
Validation auc = 0.735681
Epoch 273
Validation auc = 0.741315
Epoch 274
Loss = 2.5834e-01, PNorm = 44.5905, GNorm = 2.5606, lr_0 = 6.3280e-04
Validation auc = 0.741315
Epoch 275
Validation auc = 0.737089
Epoch 276
Validation auc = 0.736620
Epoch 277
Validation auc = 0.730986
Epoch 278
Validation auc = 0.732394
Epoch 279
Loss = 2.2140e-01, PNorm = 44.6544, GNorm = 1.7767, lr_0 = 6.2739e-04
Validation auc = 0.731925
Epoch 280
Validation auc = 0.734272
Epoch 281
Validation auc = 0.734272
Epoch 282
Validation auc = 0.731925
Epoch 283
Validation auc = 0.731925
Epoch 284
Loss = 2.2407e-01, PNorm = 44.7271, GNorm = 2.9690, lr_0 = 6.2203e-04
Validation auc = 0.734272
Epoch 285
Validation auc = 0.736150
Epoch 286
Validation auc = 0.737089
Epoch 287
Validation auc = 0.743192
Epoch 288
Validation auc = 0.737559
Epoch 289
Loss = 2.5983e-01, PNorm = 44.7923, GNorm = 2.3640, lr_0 = 6.1671e-04
Validation auc = 0.734742
Epoch 290
Validation auc = 0.731455
Epoch 291
Validation auc = 0.726761
Epoch 292
Validation auc = 0.721596
Epoch 293
Validation auc = 0.718779
Epoch 294
Loss = 2.5323e-01, PNorm = 44.8473, GNorm = 2.1431, lr_0 = 6.1144e-04
Validation auc = 0.718310
Epoch 295
Validation auc = 0.724413
Epoch 296
Validation auc = 0.726761
Epoch 297
Validation auc = 0.730047
Epoch 298
Validation auc = 0.735211
Epoch 299
Loss = 2.3729e-01, PNorm = 44.9183, GNorm = 2.0203, lr_0 = 6.0621e-04
Validation auc = 0.738967
Model 0 best validation auc = 0.777934 on epoch 15
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.779049
Ensemble test auc = 0.779049
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.661972
Epoch 1
Validation auc = 0.682160
Epoch 2
Validation auc = 0.680751
Epoch 3
Validation auc = 0.684507
Epoch 4
Loss = 1.1781e+00, PNorm = 40.6682, GNorm = 15.0723, lr_0 = 6.5000e-04
Validation auc = 0.689671
Epoch 5
Validation auc = 0.723474
Epoch 6
Validation auc = 0.690610
Epoch 7
Validation auc = 0.663380
Epoch 8
Validation auc = 0.659624
Epoch 9
Loss = 9.1711e-01, PNorm = 40.7127, GNorm = 7.3795, lr_0 = 9.9743e-04
Validation auc = 0.662911
Epoch 10
Validation auc = 0.651174
Epoch 11
Validation auc = 0.634742
Epoch 12
Validation auc = 0.628638
Epoch 13
Validation auc = 0.628169
Epoch 14
Loss = 7.2090e-01, PNorm = 40.7645, GNorm = 9.7656, lr_0 = 9.8890e-04
Validation auc = 0.651174
Epoch 15
Validation auc = 0.661502
Epoch 16
Validation auc = 0.652113
Epoch 17
Validation auc = 0.659624
Epoch 18
Validation auc = 0.669953
Epoch 19
Loss = 8.5540e-01, PNorm = 40.8080, GNorm = 7.9993, lr_0 = 9.8045e-04
Validation auc = 0.637559
Epoch 20
Validation auc = 0.624883
Epoch 21
Validation auc = 0.628169
Epoch 22
Validation auc = 0.634272
Epoch 23
Validation auc = 0.636150
Epoch 24
Loss = 6.2496e-01, PNorm = 40.8435, GNorm = 6.3456, lr_0 = 9.7207e-04
Validation auc = 0.643662
Epoch 25
Validation auc = 0.647887
Epoch 26
Validation auc = 0.653052
Epoch 27
Validation auc = 0.651643
Epoch 28
Validation auc = 0.644601
Epoch 29
Loss = 7.7754e-01, PNorm = 40.8808, GNorm = 5.8928, lr_0 = 9.6376e-04
Validation auc = 0.630986
Epoch 30
Validation auc = 0.643662
Epoch 31
Validation auc = 0.648357
Epoch 32
Validation auc = 0.630986
Epoch 33
Validation auc = 0.633803
Epoch 34
Loss = 6.5506e-01, PNorm = 40.9031, GNorm = 3.4445, lr_0 = 9.5552e-04
Validation auc = 0.630986
Epoch 35
Validation auc = 0.641315
Epoch 36
Validation auc = 0.641315
Epoch 37
Validation auc = 0.643192
Epoch 38
Validation auc = 0.650235
Epoch 39
Loss = 6.6351e-01, PNorm = 40.9500, GNorm = 5.1138, lr_0 = 9.4735e-04
Validation auc = 0.656338
Epoch 40
Validation auc = 0.653991
Epoch 41
Validation auc = 0.645540
Epoch 42
Validation auc = 0.643192
Epoch 43
Validation auc = 0.648826
Epoch 44
Loss = 5.9955e-01, PNorm = 40.9732, GNorm = 4.3191, lr_0 = 9.3925e-04
Validation auc = 0.638498
Epoch 45
Validation auc = 0.631925
Epoch 46
Validation auc = 0.623005
Epoch 47
Validation auc = 0.628169
Epoch 48
Validation auc = 0.626761
Epoch 49
Loss = 6.6602e-01, PNorm = 41.0217, GNorm = 3.8606, lr_0 = 9.3122e-04
Validation auc = 0.620657
Epoch 50
Validation auc = 0.616432
Epoch 51
Validation auc = 0.622066
Epoch 52
Validation auc = 0.633803
Epoch 53
Validation auc = 0.640376
Epoch 54
Loss = 5.4955e-01, PNorm = 41.0629, GNorm = 4.5238, lr_0 = 9.2326e-04
Validation auc = 0.633803
Epoch 55
Validation auc = 0.647887
Epoch 56
Validation auc = 0.655869
Epoch 57
Validation auc = 0.667136
Epoch 58
Validation auc = 0.675117
Epoch 59
Loss = 5.9369e-01, PNorm = 41.1049, GNorm = 4.1474, lr_0 = 9.1537e-04
Validation auc = 0.671831
Epoch 60
Validation auc = 0.663850
Epoch 61
Validation auc = 0.662441
Epoch 62
Validation auc = 0.657746
Epoch 63
Validation auc = 0.648357
Epoch 64
Loss = 5.3629e-01, PNorm = 41.1613, GNorm = 3.8278, lr_0 = 9.0754e-04
Validation auc = 0.640376
Epoch 65
Validation auc = 0.648357
Epoch 66
Validation auc = 0.652113
Epoch 67
Validation auc = 0.651174
Epoch 68
Validation auc = 0.649296
Epoch 69
Loss = 5.0683e-01, PNorm = 41.2254, GNorm = 2.8638, lr_0 = 8.9978e-04
Validation auc = 0.648357
Epoch 70
Validation auc = 0.641784
Epoch 71
Validation auc = 0.638498
Epoch 72
Validation auc = 0.629577
Epoch 73
Validation auc = 0.630516
Epoch 74
Loss = 5.5573e-01, PNorm = 41.2940, GNorm = 3.4333, lr_0 = 8.9209e-04
Validation auc = 0.637089
Epoch 75
Validation auc = 0.648357
Epoch 76
Validation auc = 0.658685
Epoch 77
Validation auc = 0.659624
Epoch 78
Validation auc = 0.653991
Epoch 79
Loss = 5.9418e-01, PNorm = 41.3519, GNorm = 2.8465, lr_0 = 8.8447e-04
Validation auc = 0.635211
Epoch 80
Validation auc = 0.623474
Epoch 81
Validation auc = 0.617840
Epoch 82
Validation auc = 0.624883
Epoch 83
Validation auc = 0.633803
Epoch 84
Loss = 4.4913e-01, PNorm = 41.4261, GNorm = 2.9812, lr_0 = 8.7691e-04
Validation auc = 0.644131
Epoch 85
Validation auc = 0.644601
Epoch 86
Validation auc = 0.630986
Epoch 87
Validation auc = 0.631455
Epoch 88
Validation auc = 0.634272
Epoch 89
Loss = 5.2753e-01, PNorm = 41.4811, GNorm = 3.4263, lr_0 = 8.6941e-04
Validation auc = 0.638028
Epoch 90
Validation auc = 0.640376
Epoch 91
Validation auc = 0.637089
Epoch 92
Validation auc = 0.642723
Epoch 93
Validation auc = 0.645070
Epoch 94
Loss = 4.1392e-01, PNorm = 41.5556, GNorm = 2.8698, lr_0 = 8.6198e-04
Validation auc = 0.648826
Epoch 95
Validation auc = 0.652582
Epoch 96
Validation auc = 0.653521
Epoch 97
Validation auc = 0.659155
Epoch 98
Validation auc = 0.654930
Epoch 99
Loss = 4.4405e-01, PNorm = 41.6218, GNorm = 3.5428, lr_0 = 8.5461e-04
Validation auc = 0.658216
Epoch 100
Validation auc = 0.647887
Epoch 101
Validation auc = 0.644601
Epoch 102
Validation auc = 0.646948
Epoch 103
Validation auc = 0.650704
Epoch 104
Loss = 5.5670e-01, PNorm = 41.6764, GNorm = 3.3436, lr_0 = 8.4730e-04
Validation auc = 0.645540
Epoch 105
Validation auc = 0.647887
Epoch 106
Validation auc = 0.657277
Epoch 107
Validation auc = 0.664319
Epoch 108
Validation auc = 0.675587
Epoch 109
Loss = 4.8457e-01, PNorm = 41.7572, GNorm = 3.0200, lr_0 = 8.4006e-04
Validation auc = 0.669014
Epoch 110
Validation auc = 0.661033
Epoch 111
Validation auc = 0.668075
Epoch 112
Validation auc = 0.671362
Epoch 113
Validation auc = 0.676056
Epoch 114
Loss = 4.9363e-01, PNorm = 41.8322, GNorm = 2.5894, lr_0 = 8.3288e-04
Validation auc = 0.684038
Epoch 115
Validation auc = 0.688732
Epoch 116
Validation auc = 0.680282
Epoch 117
Validation auc = 0.673709
Epoch 118
Validation auc = 0.661033
Epoch 119
Loss = 5.2572e-01, PNorm = 41.9128, GNorm = 2.5399, lr_0 = 8.2576e-04
Validation auc = 0.661502
Epoch 120
Validation auc = 0.662441
Epoch 121
Validation auc = 0.666197
Epoch 122
Validation auc = 0.669484
Epoch 123
Validation auc = 0.660094
Epoch 124
Loss = 4.9607e-01, PNorm = 42.0133, GNorm = 2.9986, lr_0 = 8.1870e-04
Validation auc = 0.657746
Epoch 125
Validation auc = 0.661972
Epoch 126
Validation auc = 0.678404
Epoch 127
Validation auc = 0.670423
Epoch 128
Validation auc = 0.661033
Epoch 129
Loss = 3.9704e-01, PNorm = 42.1044, GNorm = 2.5767, lr_0 = 8.1170e-04
Validation auc = 0.659155
Epoch 130
Validation auc = 0.662911
Epoch 131
Validation auc = 0.667136
Epoch 132
Validation auc = 0.672300
Epoch 133
Validation auc = 0.683099
Epoch 134
Loss = 4.0641e-01, PNorm = 42.2112, GNorm = 2.7935, lr_0 = 8.0476e-04
Validation auc = 0.700000
Epoch 135
Validation auc = 0.689671
Epoch 136
Validation auc = 0.687324
Epoch 137
Validation auc = 0.682629
Epoch 138
Validation auc = 0.678873
Epoch 139
Loss = 4.5826e-01, PNorm = 42.2806, GNorm = 4.3042, lr_0 = 7.9788e-04
Validation auc = 0.684977
Epoch 140
Validation auc = 0.693427
Epoch 141
Validation auc = 0.693897
Epoch 142
Validation auc = 0.694366
Epoch 143
Validation auc = 0.696244
Epoch 144
Loss = 4.8512e-01, PNorm = 42.3697, GNorm = 3.2741, lr_0 = 7.9106e-04
Validation auc = 0.704695
Epoch 145
Validation auc = 0.705634
Epoch 146
Validation auc = 0.713615
Epoch 147
Validation auc = 0.714554
Epoch 148
Validation auc = 0.716432
Epoch 149
Loss = 4.7121e-01, PNorm = 42.4670, GNorm = 4.2043, lr_0 = 7.8430e-04
Validation auc = 0.710798
Epoch 150
Validation auc = 0.705634
Epoch 151
Validation auc = 0.695775
Epoch 152
Validation auc = 0.694366
Epoch 153
Validation auc = 0.704695
Epoch 154
Loss = 3.6228e-01, PNorm = 42.5743, GNorm = 2.4236, lr_0 = 7.7759e-04
Validation auc = 0.704695
Epoch 155
Validation auc = 0.715493
Epoch 156
Validation auc = 0.724413
Epoch 157
Validation auc = 0.724883
Epoch 158
Validation auc = 0.724883
Epoch 159
Loss = 5.3620e-01, PNorm = 42.6870, GNorm = 4.2912, lr_0 = 7.7095e-04
Validation auc = 0.724413
Epoch 160
Validation auc = 0.727700
Epoch 161
Validation auc = 0.720188
Epoch 162
Validation auc = 0.713146
Epoch 163
Validation auc = 0.709390
Epoch 164
Loss = 3.9207e-01, PNorm = 42.7686, GNorm = 3.4823, lr_0 = 7.6436e-04
Validation auc = 0.710329
Epoch 165
Validation auc = 0.708451
Epoch 166
Validation auc = 0.709390
Epoch 167
Validation auc = 0.702347
Epoch 168
Validation auc = 0.702817
Epoch 169
Loss = 3.3011e-01, PNorm = 42.8708, GNorm = 2.2544, lr_0 = 7.5782e-04
Validation auc = 0.705164
Epoch 170
Validation auc = 0.704695
Epoch 171
Validation auc = 0.702817
Epoch 172
Validation auc = 0.707512
Epoch 173
Validation auc = 0.712676
Epoch 174
Loss = 3.4080e-01, PNorm = 42.9855, GNorm = 2.7074, lr_0 = 7.5134e-04
Validation auc = 0.709390
Epoch 175
Validation auc = 0.710329
Epoch 176
Validation auc = 0.713146
Epoch 177
Validation auc = 0.714554
Epoch 178
Validation auc = 0.715493
Epoch 179
Loss = 3.6012e-01, PNorm = 43.0959, GNorm = 2.5332, lr_0 = 7.4492e-04
Validation auc = 0.713615
Epoch 180
Validation auc = 0.719249
Epoch 181
Validation auc = 0.723474
Epoch 182
Validation auc = 0.715962
Epoch 183
Validation auc = 0.715023
Epoch 184
Loss = 4.3139e-01, PNorm = 43.1846, GNorm = 3.0004, lr_0 = 7.3855e-04
Validation auc = 0.712207
Epoch 185
Validation auc = 0.708920
Epoch 186
Validation auc = 0.715493
Epoch 187
Validation auc = 0.718310
Epoch 188
Validation auc = 0.721596
Epoch 189
Loss = 4.0971e-01, PNorm = 43.2773, GNorm = 4.3169, lr_0 = 7.3224e-04
Validation auc = 0.722066
Epoch 190
Validation auc = 0.707981
Epoch 191
Validation auc = 0.704695
Epoch 192
Validation auc = 0.713146
Epoch 193
Validation auc = 0.715493
Epoch 194
Loss = 4.0989e-01, PNorm = 43.3738, GNorm = 3.6659, lr_0 = 7.2598e-04
Validation auc = 0.717371
Epoch 195
Validation auc = 0.721127
Epoch 196
Validation auc = 0.722535
Epoch 197
Validation auc = 0.723944
Epoch 198
Validation auc = 0.718779
Epoch 199
Loss = 3.4719e-01, PNorm = 43.4653, GNorm = 2.5299, lr_0 = 7.1977e-04
Validation auc = 0.720657
Epoch 200
Validation auc = 0.715023
Epoch 201
Validation auc = 0.710329
Epoch 202
Validation auc = 0.712676
Epoch 203
Validation auc = 0.712207
Epoch 204
Loss = 3.1557e-01, PNorm = 43.5613, GNorm = 3.0541, lr_0 = 7.1362e-04
Validation auc = 0.713146
Epoch 205
Validation auc = 0.715493
Epoch 206
Validation auc = 0.715962
Epoch 207
Validation auc = 0.719249
Epoch 208
Validation auc = 0.719249
Epoch 209
Loss = 3.7296e-01, PNorm = 43.6522, GNorm = 3.5219, lr_0 = 7.0752e-04
Validation auc = 0.717840
Epoch 210
Validation auc = 0.715962
Epoch 211
Validation auc = 0.720657
Epoch 212
Validation auc = 0.719718
Epoch 213
Validation auc = 0.720188
Epoch 214
Loss = 3.6599e-01, PNorm = 43.7607, GNorm = 2.3538, lr_0 = 7.0147e-04
Validation auc = 0.723474
Epoch 215
Validation auc = 0.727230
Epoch 216
Validation auc = 0.728169
Epoch 217
Validation auc = 0.725352
Epoch 218
Validation auc = 0.726761
Epoch 219
Loss = 3.6650e-01, PNorm = 43.8413, GNorm = 2.9062, lr_0 = 6.9548e-04
Validation auc = 0.729108
Epoch 220
Validation auc = 0.730047
Epoch 221
Validation auc = 0.732394
Epoch 222
Validation auc = 0.735211
Epoch 223
Validation auc = 0.727700
Epoch 224
Loss = 3.4001e-01, PNorm = 43.9360, GNorm = 3.8500, lr_0 = 6.8953e-04
Validation auc = 0.722066
Epoch 225
Validation auc = 0.712207
Epoch 226
Validation auc = 0.714554
Epoch 227
Validation auc = 0.716901
Epoch 228
Validation auc = 0.724883
Epoch 229
Loss = 3.3092e-01, PNorm = 44.0463, GNorm = 2.6432, lr_0 = 6.8364e-04
Validation auc = 0.723944
Epoch 230
Validation auc = 0.726291
Epoch 231
Validation auc = 0.726761
Epoch 232
Validation auc = 0.729108
Epoch 233
Validation auc = 0.731455
Epoch 234
Loss = 2.2783e-01, PNorm = 44.1461, GNorm = 1.5628, lr_0 = 6.7779e-04
Validation auc = 0.730986
Epoch 235
Validation auc = 0.729577
Epoch 236
Validation auc = 0.729108
Epoch 237
Validation auc = 0.733333
Epoch 238
Validation auc = 0.735681
Epoch 239
Loss = 3.5763e-01, PNorm = 44.2374, GNorm = 2.4709, lr_0 = 6.7200e-04
Validation auc = 0.747418
Epoch 240
Validation auc = 0.762911
Epoch 241
Validation auc = 0.755399
Epoch 242
Validation auc = 0.747887
Epoch 243
Validation auc = 0.745070
Epoch 244
Loss = 3.6791e-01, PNorm = 44.3307, GNorm = 3.2160, lr_0 = 6.6625e-04
Validation auc = 0.734272
Epoch 245
Validation auc = 0.734742
Epoch 246
Validation auc = 0.729577
Epoch 247
Validation auc = 0.730047
Epoch 248
Validation auc = 0.727230
Epoch 249
Loss = 2.8694e-01, PNorm = 44.4332, GNorm = 1.6074, lr_0 = 6.6056e-04
Validation auc = 0.728638
Epoch 250
Validation auc = 0.727230
Epoch 251
Validation auc = 0.724883
Epoch 252
Validation auc = 0.723944
Epoch 253
Validation auc = 0.721596
Epoch 254
Loss = 2.2079e-01, PNorm = 44.5318, GNorm = 1.7543, lr_0 = 6.5491e-04
Validation auc = 0.722535
Epoch 255
Validation auc = 0.724883
Epoch 256
Validation auc = 0.727230
Epoch 257
Validation auc = 0.728638
Epoch 258
Validation auc = 0.724413
Epoch 259
Loss = 2.3631e-01, PNorm = 44.6401, GNorm = 3.1860, lr_0 = 6.4931e-04
Validation auc = 0.721596
Epoch 260
Validation auc = 0.723005
Epoch 261
Validation auc = 0.723005
Epoch 262
Validation auc = 0.722066
Epoch 263
Validation auc = 0.721596
Epoch 264
Loss = 2.1751e-01, PNorm = 44.7279, GNorm = 2.6049, lr_0 = 6.4376e-04
Validation auc = 0.721127
Epoch 265
Validation auc = 0.723005
Epoch 266
Validation auc = 0.724413
Epoch 267
Validation auc = 0.720657
Epoch 268
Validation auc = 0.720188
Epoch 269
Loss = 2.7057e-01, PNorm = 44.8017, GNorm = 2.4837, lr_0 = 6.3826e-04
Validation auc = 0.715023
Epoch 270
Validation auc = 0.709390
Epoch 271
Validation auc = 0.710329
Epoch 272
Validation auc = 0.707981
Epoch 273
Validation auc = 0.710329
Epoch 274
Loss = 2.6608e-01, PNorm = 44.8758, GNorm = 3.4665, lr_0 = 6.3280e-04
Validation auc = 0.713146
Epoch 275
Validation auc = 0.716901
Epoch 276
Validation auc = 0.725352
Epoch 277
Validation auc = 0.729108
Epoch 278
Validation auc = 0.737559
Epoch 279
Loss = 2.7600e-01, PNorm = 44.9589, GNorm = 2.8883, lr_0 = 6.2739e-04
Validation auc = 0.739906
Epoch 280
Validation auc = 0.737559
Epoch 281
Validation auc = 0.735211
Epoch 282
Validation auc = 0.723944
Epoch 283
Validation auc = 0.722066
Epoch 284
Loss = 2.5160e-01, PNorm = 45.0346, GNorm = 1.8325, lr_0 = 6.2203e-04
Validation auc = 0.726761
Epoch 285
Validation auc = 0.728169
Epoch 286
Validation auc = 0.723944
Epoch 287
Validation auc = 0.723005
Epoch 288
Validation auc = 0.728169
Epoch 289
Loss = 2.3684e-01, PNorm = 45.1250, GNorm = 3.1984, lr_0 = 6.1671e-04
Validation auc = 0.733803
Epoch 290
Validation auc = 0.739906
Epoch 291
Validation auc = 0.745070
Epoch 292
Validation auc = 0.749765
Epoch 293
Validation auc = 0.751643
Epoch 294
Loss = 2.8611e-01, PNorm = 45.2016, GNorm = 2.7867, lr_0 = 6.1144e-04
Validation auc = 0.745540
Epoch 295
Validation auc = 0.743192
Epoch 296
Validation auc = 0.741315
Epoch 297
Validation auc = 0.743662
Epoch 298
Validation auc = 0.737559
Epoch 299
Loss = 2.0096e-01, PNorm = 45.2722, GNorm = 1.6293, lr_0 = 6.0621e-04
Validation auc = 0.729577
Model 0 best validation auc = 0.762911 on epoch 240
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.921734
Ensemble test auc = 0.921734
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.507981
Epoch 1
Validation auc = 0.681221
Epoch 2
Validation auc = 0.676056
Epoch 3
Validation auc = 0.688263
Epoch 4
Loss = 9.2551e-01, PNorm = 40.6666, GNorm = 9.8156, lr_0 = 6.5000e-04
Validation auc = 0.712676
Epoch 5
Validation auc = 0.705634
Epoch 6
Validation auc = 0.690141
Epoch 7
Validation auc = 0.677934
Epoch 8
Validation auc = 0.653521
Epoch 9
Loss = 8.5648e-01, PNorm = 40.7162, GNorm = 9.7049, lr_0 = 9.9743e-04
Validation auc = 0.631455
Epoch 10
Validation auc = 0.638028
Epoch 11
Validation auc = 0.617840
Epoch 12
Validation auc = 0.646009
Epoch 13
Validation auc = 0.629577
Epoch 14
Loss = 9.4990e-01, PNorm = 40.7607, GNorm = 7.5925, lr_0 = 9.8890e-04
Validation auc = 0.665258
Epoch 15
Validation auc = 0.697653
Epoch 16
Validation auc = 0.716901
Epoch 17
Validation auc = 0.678404
Epoch 18
Validation auc = 0.682160
Epoch 19
Loss = 7.0413e-01, PNorm = 40.7954, GNorm = 5.6336, lr_0 = 9.8045e-04
Validation auc = 0.703756
Epoch 20
Validation auc = 0.786854
Epoch 21
Validation auc = 0.835681
Epoch 22
Validation auc = 0.815023
Epoch 23
Validation auc = 0.711268
Epoch 24
Loss = 6.5052e-01, PNorm = 40.8276, GNorm = 5.5954, lr_0 = 9.7207e-04
Validation auc = 0.698122
Epoch 25
Validation auc = 0.700939
Epoch 26
Validation auc = 0.710798
Epoch 27
Validation auc = 0.694366
Epoch 28
Validation auc = 0.687793
Epoch 29
Loss = 5.5701e-01, PNorm = 40.8621, GNorm = 5.6477, lr_0 = 9.6376e-04
Validation auc = 0.676056
Epoch 30
Validation auc = 0.666197
Epoch 31
Validation auc = 0.667606
Epoch 32
Validation auc = 0.688732
Epoch 33
Validation auc = 0.731925
Epoch 34
Loss = 6.4863e-01, PNorm = 40.8961, GNorm = 5.7479, lr_0 = 9.5552e-04
Validation auc = 0.715493
Epoch 35
Validation auc = 0.692958
Epoch 36
Validation auc = 0.692488
Epoch 37
Validation auc = 0.688732
Epoch 38
Validation auc = 0.714554
Epoch 39
Loss = 5.7298e-01, PNorm = 40.9312, GNorm = 4.5262, lr_0 = 9.4735e-04
Validation auc = 0.747418
Epoch 40
Validation auc = 0.759155
Epoch 41
Validation auc = 0.740376
Epoch 42
Validation auc = 0.721127
Epoch 43
Validation auc = 0.726291
Epoch 44
Loss = 5.9993e-01, PNorm = 40.9665, GNorm = 5.0135, lr_0 = 9.3925e-04
Validation auc = 0.739437
Epoch 45
Validation auc = 0.732394
Epoch 46
Validation auc = 0.705634
Epoch 47
Validation auc = 0.694836
Epoch 48
Validation auc = 0.690141
Epoch 49
Loss = 5.9951e-01, PNorm = 41.0116, GNorm = 4.7488, lr_0 = 9.3122e-04
Validation auc = 0.686385
Epoch 50
Validation auc = 0.687324
Epoch 51
Validation auc = 0.692488
Epoch 52
Validation auc = 0.690610
Epoch 53
Validation auc = 0.684977
Epoch 54
Loss = 5.1889e-01, PNorm = 41.0583, GNorm = 3.1189, lr_0 = 9.2326e-04
Validation auc = 0.687324
Epoch 55
Validation auc = 0.689202
Epoch 56
Validation auc = 0.685915
Epoch 57
Validation auc = 0.681690
Epoch 58
Validation auc = 0.683568
Epoch 59
Loss = 6.2089e-01, PNorm = 41.1031, GNorm = 4.3007, lr_0 = 9.1537e-04
Validation auc = 0.656808
Epoch 60
Validation auc = 0.625822
Epoch 61
Validation auc = 0.605634
Epoch 62
Validation auc = 0.601408
Epoch 63
Validation auc = 0.603286
Epoch 64
Loss = 5.4972e-01, PNorm = 41.1586, GNorm = 4.7985, lr_0 = 9.0754e-04
Validation auc = 0.607042
Epoch 65
Validation auc = 0.615493
Epoch 66
Validation auc = 0.622535
Epoch 67
Validation auc = 0.623005
Epoch 68
Validation auc = 0.616432
Epoch 69
Loss = 5.5597e-01, PNorm = 41.2086, GNorm = 2.8667, lr_0 = 8.9978e-04
Validation auc = 0.607981
Epoch 70
Validation auc = 0.601408
Epoch 71
Validation auc = 0.599531
Epoch 72
Validation auc = 0.602817
Epoch 73
Validation auc = 0.598122
Epoch 74
Loss = 5.6000e-01, PNorm = 41.2612, GNorm = 3.0247, lr_0 = 8.9209e-04
Validation auc = 0.609390
Epoch 75
Validation auc = 0.622535
Epoch 76
Validation auc = 0.626761
Epoch 77
Validation auc = 0.623474
Epoch 78
Validation auc = 0.613146
Epoch 79
Loss = 4.7712e-01, PNorm = 41.3117, GNorm = 3.6181, lr_0 = 8.8447e-04
Validation auc = 0.623005
Epoch 80
Validation auc = 0.630047
Epoch 81
Validation auc = 0.631455
Epoch 82
Validation auc = 0.638498
Epoch 83
Validation auc = 0.645540
Epoch 84
Loss = 5.3522e-01, PNorm = 41.3727, GNorm = 2.7674, lr_0 = 8.7691e-04
Validation auc = 0.649765
Epoch 85
Validation auc = 0.642254
Epoch 86
Validation auc = 0.646009
Epoch 87
Validation auc = 0.657746
Epoch 88
Validation auc = 0.670423
Epoch 89
Loss = 5.7134e-01, PNorm = 41.4322, GNorm = 3.1952, lr_0 = 8.6941e-04
Validation auc = 0.676526
Epoch 90
Validation auc = 0.684038
Epoch 91
Validation auc = 0.690610
Epoch 92
Validation auc = 0.699061
Epoch 93
Validation auc = 0.698592
Epoch 94
Loss = 4.8302e-01, PNorm = 41.5126, GNorm = 3.6096, lr_0 = 8.6198e-04
Validation auc = 0.701408
Epoch 95
Validation auc = 0.696714
Epoch 96
Validation auc = 0.689671
Epoch 97
Validation auc = 0.686385
Epoch 98
Validation auc = 0.681690
Epoch 99
Loss = 4.8676e-01, PNorm = 41.5784, GNorm = 3.5674, lr_0 = 8.5461e-04
Validation auc = 0.688732
Epoch 100
Validation auc = 0.700000
Epoch 101
Validation auc = 0.707981
Epoch 102
Validation auc = 0.700939
Epoch 103
Validation auc = 0.700000
Epoch 104
Loss = 5.0546e-01, PNorm = 41.6618, GNorm = 3.1421, lr_0 = 8.4730e-04
Validation auc = 0.696714
Epoch 105
Validation auc = 0.693897
Epoch 106
Validation auc = 0.690141
Epoch 107
Validation auc = 0.673709
Epoch 108
Validation auc = 0.663850
Epoch 109
Loss = 4.4904e-01, PNorm = 41.7526, GNorm = 3.4140, lr_0 = 8.4006e-04
Validation auc = 0.666667
Epoch 110
Validation auc = 0.659155
Epoch 111
Validation auc = 0.646479
Epoch 112
Validation auc = 0.663850
Epoch 113
Validation auc = 0.661972
Epoch 114
Loss = 6.0960e-01, PNorm = 41.8488, GNorm = 4.5595, lr_0 = 8.3288e-04
Validation auc = 0.660094
Epoch 115
Validation auc = 0.659155
Epoch 116
Validation auc = 0.658216
Epoch 117
Validation auc = 0.683099
Epoch 118
Validation auc = 0.678873
Epoch 119
Loss = 4.0929e-01, PNorm = 41.9473, GNorm = 2.5357, lr_0 = 8.2576e-04
Validation auc = 0.666667
Epoch 120
Validation auc = 0.644601
Epoch 121
Validation auc = 0.642723
Epoch 122
Validation auc = 0.639437
Epoch 123
Validation auc = 0.646479
Epoch 124
Loss = 5.2056e-01, PNorm = 42.0355, GNorm = 3.8653, lr_0 = 8.1870e-04
Validation auc = 0.666197
Epoch 125
Validation auc = 0.681221
Epoch 126
Validation auc = 0.693897
Epoch 127
Validation auc = 0.696714
Epoch 128
Validation auc = 0.694836
Epoch 129
Loss = 4.1412e-01, PNorm = 42.1342, GNorm = 1.8463, lr_0 = 8.1170e-04
Validation auc = 0.692019
Epoch 130
Validation auc = 0.685915
Epoch 131
Validation auc = 0.703756
Epoch 132
Validation auc = 0.708920
Epoch 133
Validation auc = 0.719718
Epoch 134
Loss = 4.9161e-01, PNorm = 42.2235, GNorm = 3.0690, lr_0 = 8.0476e-04
Validation auc = 0.724413
Epoch 135
Validation auc = 0.727700
Epoch 136
Validation auc = 0.729108
Epoch 137
Validation auc = 0.722535
Epoch 138
Validation auc = 0.710329
Epoch 139
Loss = 4.2085e-01, PNorm = 42.3326, GNorm = 4.0519, lr_0 = 7.9788e-04
Validation auc = 0.712207
Epoch 140
Validation auc = 0.708451
Epoch 141
Validation auc = 0.703286
Epoch 142
Validation auc = 0.721596
Epoch 143
Validation auc = 0.714554
Epoch 144
Loss = 4.4986e-01, PNorm = 42.4493, GNorm = 2.5090, lr_0 = 7.9106e-04
Validation auc = 0.726291
Epoch 145
Validation auc = 0.722535
Epoch 146
Validation auc = 0.721596
Epoch 147
Validation auc = 0.726761
Epoch 148
Validation auc = 0.735681
Epoch 149
Loss = 4.9168e-01, PNorm = 42.5552, GNorm = 3.1605, lr_0 = 7.8430e-04
Validation auc = 0.733803
Epoch 150
Validation auc = 0.739906
Epoch 151
Validation auc = 0.741784
Epoch 152
Validation auc = 0.735211
Epoch 153
Validation auc = 0.730516
Epoch 154
Loss = 3.2871e-01, PNorm = 42.6718, GNorm = 3.0211, lr_0 = 7.7759e-04
Validation auc = 0.724413
Epoch 155
Validation auc = 0.725352
Epoch 156
Validation auc = 0.732394
Epoch 157
Validation auc = 0.731455
Epoch 158
Validation auc = 0.750704
Epoch 159
Loss = 3.9675e-01, PNorm = 42.7862, GNorm = 4.1472, lr_0 = 7.7095e-04
Validation auc = 0.751643
Epoch 160
Validation auc = 0.734742
Epoch 161
Validation auc = 0.731925
Epoch 162
Validation auc = 0.730516
Epoch 163
Validation auc = 0.727230
Epoch 164
Loss = 4.0452e-01, PNorm = 42.8710, GNorm = 4.3885, lr_0 = 7.6436e-04
Validation auc = 0.727700
Epoch 165
Validation auc = 0.730047
Epoch 166
Validation auc = 0.739437
Epoch 167
Validation auc = 0.747887
Epoch 168
Validation auc = 0.748357
Epoch 169
Loss = 3.0051e-01, PNorm = 42.9763, GNorm = 2.3212, lr_0 = 7.5782e-04
Validation auc = 0.740376
Epoch 170
Validation auc = 0.731455
Epoch 171
Validation auc = 0.723944
Epoch 172
Validation auc = 0.713146
Epoch 173
Validation auc = 0.707512
Epoch 174
Loss = 3.6762e-01, PNorm = 43.0772, GNorm = 3.2309, lr_0 = 7.5134e-04
Validation auc = 0.711268
Epoch 175
Validation auc = 0.719249
Epoch 176
Validation auc = 0.721127
Epoch 177
Validation auc = 0.733333
Epoch 178
Validation auc = 0.739437
Epoch 179
Loss = 3.5989e-01, PNorm = 43.1597, GNorm = 2.8168, lr_0 = 7.4492e-04
Validation auc = 0.739906
Epoch 180
Validation auc = 0.743662
Epoch 181
Validation auc = 0.745070
Epoch 182
Validation auc = 0.746948
Epoch 183
Validation auc = 0.733803
Epoch 184
Loss = 4.0502e-01, PNorm = 43.2428, GNorm = 2.9526, lr_0 = 7.3855e-04
Validation auc = 0.723944
Epoch 185
Validation auc = 0.721127
Epoch 186
Validation auc = 0.718779
Epoch 187
Validation auc = 0.722066
Epoch 188
Validation auc = 0.733333
Epoch 189
Loss = 2.6426e-01, PNorm = 43.3387, GNorm = 1.7376, lr_0 = 7.3224e-04
Validation auc = 0.746009
Epoch 190
Validation auc = 0.753521
Epoch 191
Validation auc = 0.756808
Epoch 192
Validation auc = 0.765258
Epoch 193
Validation auc = 0.761033
Epoch 194
Loss = 2.9796e-01, PNorm = 43.4258, GNorm = 2.2773, lr_0 = 7.2598e-04
Validation auc = 0.762441
Epoch 195
Validation auc = 0.758216
Epoch 196
Validation auc = 0.750704
Epoch 197
Validation auc = 0.745070
Epoch 198
Validation auc = 0.740376
Epoch 199
Loss = 3.3403e-01, PNorm = 43.5011, GNorm = 1.9868, lr_0 = 7.1977e-04
Validation auc = 0.744131
Epoch 200
Validation auc = 0.746948
Epoch 201
Validation auc = 0.750704
Epoch 202
Validation auc = 0.759155
Epoch 203
Validation auc = 0.765258
Epoch 204
Loss = 2.7899e-01, PNorm = 43.5770, GNorm = 2.0935, lr_0 = 7.1362e-04
Validation auc = 0.769484
Epoch 205
Validation auc = 0.761502
Epoch 206
Validation auc = 0.749765
Epoch 207
Validation auc = 0.749296
Epoch 208
Validation auc = 0.748826
Epoch 209
Loss = 3.3478e-01, PNorm = 43.6682, GNorm = 2.7318, lr_0 = 7.0752e-04
Validation auc = 0.747418
Epoch 210
Validation auc = 0.747887
Epoch 211
Validation auc = 0.749296
Epoch 212
Validation auc = 0.753521
Epoch 213
Validation auc = 0.757277
Epoch 214
Loss = 3.6373e-01, PNorm = 43.7502, GNorm = 3.2662, lr_0 = 7.0147e-04
Validation auc = 0.754930
Epoch 215
Validation auc = 0.748826
Epoch 216
Validation auc = 0.739437
Epoch 217
Validation auc = 0.734272
Epoch 218
Validation auc = 0.733333
Epoch 219
Loss = 2.8889e-01, PNorm = 43.8212, GNorm = 1.4596, lr_0 = 6.9548e-04
Validation auc = 0.729577
Epoch 220
Validation auc = 0.721596
Epoch 221
Validation auc = 0.717840
Epoch 222
Validation auc = 0.715962
Epoch 223
Validation auc = 0.714554
Epoch 224
Loss = 2.7272e-01, PNorm = 43.8994, GNorm = 3.6426, lr_0 = 6.8953e-04
Validation auc = 0.718310
Epoch 225
Validation auc = 0.723005
Epoch 226
Validation auc = 0.727700
Epoch 227
Validation auc = 0.737559
Epoch 228
Validation auc = 0.740376
Epoch 229
Loss = 3.6931e-01, PNorm = 43.9625, GNorm = 2.3944, lr_0 = 6.8364e-04
Validation auc = 0.741315
Epoch 230
Validation auc = 0.744131
Epoch 231
Validation auc = 0.753521
Epoch 232
Validation auc = 0.766667
Epoch 233
Validation auc = 0.768545
Epoch 234
Loss = 4.3187e-01, PNorm = 44.0352, GNorm = 2.0812, lr_0 = 6.7779e-04
Validation auc = 0.762441
Epoch 235
Validation auc = 0.765258
Epoch 236
Validation auc = 0.766197
Epoch 237
Validation auc = 0.767136
Epoch 238
Validation auc = 0.759155
Epoch 239
Loss = 3.2616e-01, PNorm = 44.1099, GNorm = 3.8610, lr_0 = 6.7200e-04
Validation auc = 0.758685
Epoch 240
Validation auc = 0.753521
Epoch 241
Validation auc = 0.759624
Epoch 242
Validation auc = 0.758685
Epoch 243
Validation auc = 0.751174
Epoch 244
Loss = 2.8388e-01, PNorm = 44.1916, GNorm = 1.6452, lr_0 = 6.6625e-04
Validation auc = 0.747887
Epoch 245
Validation auc = 0.735211
Epoch 246
Validation auc = 0.732864
Epoch 247
Validation auc = 0.732864
Epoch 248
Validation auc = 0.737089
Epoch 249
Loss = 3.0000e-01, PNorm = 44.2691, GNorm = 2.0629, lr_0 = 6.6056e-04
Validation auc = 0.737559
Epoch 250
Validation auc = 0.740376
Epoch 251
Validation auc = 0.745070
Epoch 252
Validation auc = 0.740845
Epoch 253
Validation auc = 0.740376
Epoch 254
Loss = 3.1042e-01, PNorm = 44.3411, GNorm = 3.2057, lr_0 = 6.5491e-04
Validation auc = 0.740845
Epoch 255
Validation auc = 0.742254
Epoch 256
Validation auc = 0.739437
Epoch 257
Validation auc = 0.736620
Epoch 258
Validation auc = 0.736620
Epoch 259
Loss = 2.8796e-01, PNorm = 44.4145, GNorm = 2.3813, lr_0 = 6.4931e-04
Validation auc = 0.732864
Epoch 260
Validation auc = 0.738498
Epoch 261
Validation auc = 0.748826
Epoch 262
Validation auc = 0.745540
Epoch 263
Validation auc = 0.738028
Epoch 264
Loss = 2.8911e-01, PNorm = 44.4792, GNorm = 2.8896, lr_0 = 6.4376e-04
Validation auc = 0.734742
Epoch 265
Validation auc = 0.731925
Epoch 266
Validation auc = 0.735211
Epoch 267
Validation auc = 0.735211
Epoch 268
Validation auc = 0.735681
Epoch 269
Loss = 2.7430e-01, PNorm = 44.5443, GNorm = 1.3776, lr_0 = 6.3826e-04
Validation auc = 0.737559
Epoch 270
Validation auc = 0.739906
Epoch 271
Validation auc = 0.740845
Epoch 272
Validation auc = 0.743192
Epoch 273
Validation auc = 0.744601
Epoch 274
Loss = 2.1225e-01, PNorm = 44.6094, GNorm = 1.6614, lr_0 = 6.3280e-04
Validation auc = 0.739437
Epoch 275
Validation auc = 0.738028
Epoch 276
Validation auc = 0.741315
Epoch 277
Validation auc = 0.744601
Epoch 278
Validation auc = 0.745540
Epoch 279
Loss = 2.6938e-01, PNorm = 44.6797, GNorm = 1.4283, lr_0 = 6.2739e-04
Validation auc = 0.737559
Epoch 280
Validation auc = 0.730516
Epoch 281
Validation auc = 0.730516
Epoch 282
Validation auc = 0.737089
Epoch 283
Validation auc = 0.753521
Epoch 284
Loss = 3.3574e-01, PNorm = 44.7613, GNorm = 3.8172, lr_0 = 6.2203e-04
Validation auc = 0.755869
Epoch 285
Validation auc = 0.750235
Epoch 286
Validation auc = 0.746948
Epoch 287
Validation auc = 0.748357
Epoch 288
Validation auc = 0.751174
Epoch 289
Loss = 2.5106e-01, PNorm = 44.8312, GNorm = 1.5080, lr_0 = 6.1671e-04
Validation auc = 0.758216
Epoch 290
Validation auc = 0.756338
Epoch 291
Validation auc = 0.758216
Epoch 292
Validation auc = 0.755399
Epoch 293
Validation auc = 0.751174
Epoch 294
Loss = 2.6446e-01, PNorm = 44.9100, GNorm = 2.0535, lr_0 = 6.1144e-04
Validation auc = 0.753052
Epoch 295
Validation auc = 0.746948
Epoch 296
Validation auc = 0.746479
Epoch 297
Validation auc = 0.744131
Epoch 298
Validation auc = 0.736620
Epoch 299
Loss = 2.6127e-01, PNorm = 44.9815, GNorm = 4.4577, lr_0 = 6.0621e-04
Validation auc = 0.735681
Model 0 best validation auc = 0.835681 on epoch 21
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.797110
Ensemble test auc = 0.797110
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.511268
Epoch 1
Validation auc = 0.688263
Epoch 2
Validation auc = 0.684038
Epoch 3
Validation auc = 0.703286
Epoch 4
Loss = 1.0335e+00, PNorm = 40.6655, GNorm = 13.8268, lr_0 = 6.5000e-04
Validation auc = 0.715962
Epoch 5
Validation auc = 0.721596
Epoch 6
Validation auc = 0.666667
Epoch 7
Validation auc = 0.661972
Epoch 8
Validation auc = 0.644601
Epoch 9
Loss = 1.0849e+00, PNorm = 40.7127, GNorm = 12.4416, lr_0 = 9.9743e-04
Validation auc = 0.644601
Epoch 10
Validation auc = 0.657746
Epoch 11
Validation auc = 0.652582
Epoch 12
Validation auc = 0.650704
Epoch 13
Validation auc = 0.638967
Epoch 14
Loss = 7.3438e-01, PNorm = 40.7623, GNorm = 6.9317, lr_0 = 9.8890e-04
Validation auc = 0.639437
Epoch 15
Validation auc = 0.650704
Epoch 16
Validation auc = 0.652113
Epoch 17
Validation auc = 0.650704
Epoch 18
Validation auc = 0.688263
Epoch 19
Loss = 7.3504e-01, PNorm = 40.8110, GNorm = 5.9748, lr_0 = 9.8045e-04
Validation auc = 0.710798
Epoch 20
Validation auc = 0.697183
Epoch 21
Validation auc = 0.694366
Epoch 22
Validation auc = 0.687324
Epoch 23
Validation auc = 0.689202
Epoch 24
Loss = 7.7267e-01, PNorm = 40.8503, GNorm = 8.7644, lr_0 = 9.7207e-04
Validation auc = 0.704225
Epoch 25
Validation auc = 0.701878
Epoch 26
Validation auc = 0.707042
Epoch 27
Validation auc = 0.713615
Epoch 28
Validation auc = 0.696714
Epoch 29
Loss = 6.7391e-01, PNorm = 40.8792, GNorm = 6.4853, lr_0 = 9.6376e-04
Validation auc = 0.722535
Epoch 30
Validation auc = 0.664319
Epoch 31
Validation auc = 0.660563
Epoch 32
Validation auc = 0.658685
Epoch 33
Validation auc = 0.647887
Epoch 34
Loss = 5.8075e-01, PNorm = 40.9269, GNorm = 3.7780, lr_0 = 9.5552e-04
Validation auc = 0.654460
Epoch 35
Validation auc = 0.639437
Epoch 36
Validation auc = 0.624413
Epoch 37
Validation auc = 0.617371
Epoch 38
Validation auc = 0.606573
Epoch 39
Loss = 6.7670e-01, PNorm = 40.9526, GNorm = 5.6055, lr_0 = 9.4735e-04
Validation auc = 0.609390
Epoch 40
Validation auc = 0.616901
Epoch 41
Validation auc = 0.643192
Epoch 42
Validation auc = 0.664789
Epoch 43
Validation auc = 0.661972
Epoch 44
Loss = 6.2557e-01, PNorm = 40.9860, GNorm = 3.8241, lr_0 = 9.3925e-04
Validation auc = 0.649765
Epoch 45
Validation auc = 0.641315
Epoch 46
Validation auc = 0.636150
Epoch 47
Validation auc = 0.642723
Epoch 48
Validation auc = 0.644131
Epoch 49
Loss = 5.5161e-01, PNorm = 41.0275, GNorm = 3.8299, lr_0 = 9.3122e-04
Validation auc = 0.644601
Epoch 50
Validation auc = 0.640376
Epoch 51
Validation auc = 0.643192
Epoch 52
Validation auc = 0.649296
Epoch 53
Validation auc = 0.645540
Epoch 54
Loss = 6.3878e-01, PNorm = 41.0698, GNorm = 5.2715, lr_0 = 9.2326e-04
Validation auc = 0.638028
Epoch 55
Validation auc = 0.634272
Epoch 56
Validation auc = 0.642723
Epoch 57
Validation auc = 0.644131
Epoch 58
Validation auc = 0.650235
Epoch 59
Loss = 6.4173e-01, PNorm = 41.1079, GNorm = 3.5508, lr_0 = 9.1537e-04
Validation auc = 0.684507
Epoch 60
Validation auc = 0.685915
Epoch 61
Validation auc = 0.673239
Epoch 62
Validation auc = 0.672770
Epoch 63
Validation auc = 0.682160
Epoch 64
Loss = 5.6578e-01, PNorm = 41.1633, GNorm = 4.0758, lr_0 = 9.0754e-04
Validation auc = 0.687793
Epoch 65
Validation auc = 0.680282
Epoch 66
Validation auc = 0.663850
Epoch 67
Validation auc = 0.649765
Epoch 68
Validation auc = 0.642723
Epoch 69
Loss = 5.6712e-01, PNorm = 41.2104, GNorm = 3.5649, lr_0 = 8.9978e-04
Validation auc = 0.642254
Epoch 70
Validation auc = 0.647418
Epoch 71
Validation auc = 0.648357
Epoch 72
Validation auc = 0.644601
Epoch 73
Validation auc = 0.651174
Epoch 74
Loss = 5.9027e-01, PNorm = 41.2907, GNorm = 3.5333, lr_0 = 8.9209e-04
Validation auc = 0.653052
Epoch 75
Validation auc = 0.652582
Epoch 76
Validation auc = 0.661502
Epoch 77
Validation auc = 0.674178
Epoch 78
Validation auc = 0.690141
Epoch 79
Loss = 5.9295e-01, PNorm = 41.3316, GNorm = 4.2430, lr_0 = 8.8447e-04
Validation auc = 0.691549
Epoch 80
Validation auc = 0.694836
Epoch 81
Validation auc = 0.668545
Epoch 82
Validation auc = 0.638498
Epoch 83
Validation auc = 0.643662
Epoch 84
Loss = 5.5442e-01, PNorm = 41.4069, GNorm = 5.3977, lr_0 = 8.7691e-04
Validation auc = 0.655869
Epoch 85
Validation auc = 0.662441
Epoch 86
Validation auc = 0.666667
Epoch 87
Validation auc = 0.666197
Epoch 88
Validation auc = 0.661033
Epoch 89
Loss = 4.4261e-01, PNorm = 41.4816, GNorm = 3.2324, lr_0 = 8.6941e-04
Validation auc = 0.651643
Epoch 90
Validation auc = 0.640845
Epoch 91
Validation auc = 0.631925
Epoch 92
Validation auc = 0.615962
Epoch 93
Validation auc = 0.615962
Epoch 94
Loss = 4.8953e-01, PNorm = 41.5556, GNorm = 3.5319, lr_0 = 8.6198e-04
Validation auc = 0.620188
Epoch 95
Validation auc = 0.621596
Epoch 96
Validation auc = 0.619718
Epoch 97
Validation auc = 0.608451
Epoch 98
Validation auc = 0.606573
Epoch 99
Loss = 5.8039e-01, PNorm = 41.6272, GNorm = 3.8166, lr_0 = 8.5461e-04
Validation auc = 0.619249
Epoch 100
Validation auc = 0.626761
Epoch 101
Validation auc = 0.645070
Epoch 102
Validation auc = 0.664319
Epoch 103
Validation auc = 0.664789
Epoch 104
Loss = 4.5526e-01, PNorm = 41.7101, GNorm = 2.8701, lr_0 = 8.4730e-04
Validation auc = 0.666197
Epoch 105
Validation auc = 0.659155
Epoch 106
Validation auc = 0.644131
Epoch 107
Validation auc = 0.635211
Epoch 108
Validation auc = 0.638498
Epoch 109
Loss = 4.9369e-01, PNorm = 41.7676, GNorm = 2.2677, lr_0 = 8.4006e-04
Validation auc = 0.644131
Epoch 110
Validation auc = 0.650704
Epoch 111
Validation auc = 0.649765
Epoch 112
Validation auc = 0.638498
Epoch 113
Validation auc = 0.643192
Epoch 114
Loss = 4.8779e-01, PNorm = 41.8602, GNorm = 3.7279, lr_0 = 8.3288e-04
Validation auc = 0.652113
Epoch 115
Validation auc = 0.669014
Epoch 116
Validation auc = 0.663380
Epoch 117
Validation auc = 0.668545
Epoch 118
Validation auc = 0.668075
Epoch 119
Loss = 5.8264e-01, PNorm = 41.9553, GNorm = 2.3273, lr_0 = 8.2576e-04
Validation auc = 0.661033
Epoch 120
Validation auc = 0.660563
Epoch 121
Validation auc = 0.671362
Epoch 122
Validation auc = 0.676056
Epoch 123
Validation auc = 0.680282
Epoch 124
Loss = 4.0334e-01, PNorm = 42.0540, GNorm = 3.0817, lr_0 = 8.1870e-04
Validation auc = 0.671831
Epoch 125
Validation auc = 0.668545
Epoch 126
Validation auc = 0.666667
Epoch 127
Validation auc = 0.673239
Epoch 128
Validation auc = 0.670892
Epoch 129
Loss = 4.8197e-01, PNorm = 42.1586, GNorm = 2.7118, lr_0 = 8.1170e-04
Validation auc = 0.683099
Epoch 130
Validation auc = 0.700939
Epoch 131
Validation auc = 0.701408
Epoch 132
Validation auc = 0.700000
Epoch 133
Validation auc = 0.701878
Epoch 134
Loss = 3.7057e-01, PNorm = 42.2632, GNorm = 3.0221, lr_0 = 8.0476e-04
Validation auc = 0.706573
Epoch 135
Validation auc = 0.708920
Epoch 136
Validation auc = 0.703286
Epoch 137
Validation auc = 0.708920
Epoch 138
Validation auc = 0.713615
Epoch 139
Loss = 4.2169e-01, PNorm = 42.3812, GNorm = 2.1885, lr_0 = 7.9788e-04
Validation auc = 0.709390
Epoch 140
Validation auc = 0.707042
Epoch 141
Validation auc = 0.700000
Epoch 142
Validation auc = 0.701408
Epoch 143
Validation auc = 0.704225
Epoch 144
Loss = 3.9695e-01, PNorm = 42.4665, GNorm = 2.3187, lr_0 = 7.9106e-04
Validation auc = 0.709859
Epoch 145
Validation auc = 0.714554
Epoch 146
Validation auc = 0.710798
Epoch 147
Validation auc = 0.716901
Epoch 148
Validation auc = 0.714554
Epoch 149
Loss = 4.3793e-01, PNorm = 42.5498, GNorm = 3.9545, lr_0 = 7.8430e-04
Validation auc = 0.723474
Epoch 150
Validation auc = 0.728169
Epoch 151
Validation auc = 0.734742
Epoch 152
Validation auc = 0.738967
Epoch 153
Validation auc = 0.738498
Epoch 154
Loss = 4.4117e-01, PNorm = 42.6555, GNorm = 2.8707, lr_0 = 7.7759e-04
Validation auc = 0.727700
Epoch 155
Validation auc = 0.724883
Epoch 156
Validation auc = 0.716432
Epoch 157
Validation auc = 0.714554
Epoch 158
Validation auc = 0.703286
Epoch 159
Loss = 4.5156e-01, PNorm = 42.7668, GNorm = 3.4066, lr_0 = 7.7095e-04
Validation auc = 0.707042
Epoch 160
Validation auc = 0.721596
Epoch 161
Validation auc = 0.730047
Epoch 162
Validation auc = 0.734742
Epoch 163
Validation auc = 0.735681
Epoch 164
Loss = 4.3522e-01, PNorm = 42.8561, GNorm = 3.6832, lr_0 = 7.6436e-04
Validation auc = 0.714085
Epoch 165
Validation auc = 0.702817
Epoch 166
Validation auc = 0.703286
Epoch 167
Validation auc = 0.693897
Epoch 168
Validation auc = 0.693427
Epoch 169
Loss = 3.3454e-01, PNorm = 42.9635, GNorm = 2.6490, lr_0 = 7.5782e-04
Validation auc = 0.694836
Epoch 170
Validation auc = 0.697183
Epoch 171
Validation auc = 0.704695
Epoch 172
Validation auc = 0.713146
Epoch 173
Validation auc = 0.713146
Epoch 174
Loss = 3.3167e-01, PNorm = 43.0684, GNorm = 1.9595, lr_0 = 7.5134e-04
Validation auc = 0.719718
Epoch 175
Validation auc = 0.718310
Epoch 176
Validation auc = 0.714085
Epoch 177
Validation auc = 0.713146
Epoch 178
Validation auc = 0.712207
Epoch 179
Loss = 3.0406e-01, PNorm = 43.1787, GNorm = 2.6592, lr_0 = 7.4492e-04
Validation auc = 0.728638
Epoch 180
Validation auc = 0.746948
Epoch 181
Validation auc = 0.751643
Epoch 182
Validation auc = 0.740376
Epoch 183
Validation auc = 0.738967
Epoch 184
Loss = 4.1302e-01, PNorm = 43.2710, GNorm = 2.9524, lr_0 = 7.3855e-04
Validation auc = 0.746479
Epoch 185
Validation auc = 0.748826
Epoch 186
Validation auc = 0.740376
Epoch 187
Validation auc = 0.725822
Epoch 188
Validation auc = 0.705634
Epoch 189
Loss = 4.0835e-01, PNorm = 43.3506, GNorm = 2.3178, lr_0 = 7.3224e-04
Validation auc = 0.715493
Epoch 190
Validation auc = 0.719718
Epoch 191
Validation auc = 0.719718
Epoch 192
Validation auc = 0.724883
Epoch 193
Validation auc = 0.724883
Epoch 194
Loss = 3.8194e-01, PNorm = 43.4408, GNorm = 1.8036, lr_0 = 7.2598e-04
Validation auc = 0.723944
Epoch 195
Validation auc = 0.718310
Epoch 196
Validation auc = 0.726761
Epoch 197
Validation auc = 0.726291
Epoch 198
Validation auc = 0.729577
Epoch 199
Loss = 3.3246e-01, PNorm = 43.5347, GNorm = 2.2120, lr_0 = 7.1977e-04
Validation auc = 0.719718
Epoch 200
Validation auc = 0.710798
Epoch 201
Validation auc = 0.699531
Epoch 202
Validation auc = 0.708451
Epoch 203
Validation auc = 0.709859
Epoch 204
Loss = 3.4608e-01, PNorm = 43.6281, GNorm = 3.8036, lr_0 = 7.1362e-04
Validation auc = 0.719249
Epoch 205
Validation auc = 0.724883
Epoch 206
Validation auc = 0.726291
Epoch 207
Validation auc = 0.726291
Epoch 208
Validation auc = 0.727230
Epoch 209
Loss = 3.7296e-01, PNorm = 43.7045, GNorm = 2.8805, lr_0 = 7.0752e-04
Validation auc = 0.727700
Epoch 210
Validation auc = 0.725352
Epoch 211
Validation auc = 0.735211
Epoch 212
Validation auc = 0.730986
Epoch 213
Validation auc = 0.724413
Epoch 214
Loss = 3.0332e-01, PNorm = 43.8134, GNorm = 1.9838, lr_0 = 7.0147e-04
Validation auc = 0.725352
Epoch 215
Validation auc = 0.721596
Epoch 216
Validation auc = 0.724883
Epoch 217
Validation auc = 0.720657
Epoch 218
Validation auc = 0.724413
Epoch 219
Loss = 3.6710e-01, PNorm = 43.9164, GNorm = 3.2902, lr_0 = 6.9548e-04
Validation auc = 0.720657
Epoch 220
Validation auc = 0.725822
Epoch 221
Validation auc = 0.725822
Epoch 222
Validation auc = 0.717371
Epoch 223
Validation auc = 0.719249
Epoch 224
Loss = 3.0545e-01, PNorm = 43.9850, GNorm = 2.4454, lr_0 = 6.8953e-04
Validation auc = 0.722535
Epoch 225
Validation auc = 0.718779
Epoch 226
Validation auc = 0.720657
Epoch 227
Validation auc = 0.721127
Epoch 228
Validation auc = 0.722066
Epoch 229
Loss = 2.8623e-01, PNorm = 44.0787, GNorm = 2.3642, lr_0 = 6.8364e-04
Validation auc = 0.725352
Epoch 230
Validation auc = 0.726291
Epoch 231
Validation auc = 0.723944
Epoch 232
Validation auc = 0.727230
Epoch 233
Validation auc = 0.727230
Epoch 234
Loss = 3.4498e-01, PNorm = 44.1853, GNorm = 4.1308, lr_0 = 6.7779e-04
Validation auc = 0.724883
Epoch 235
Validation auc = 0.720188
Epoch 236
Validation auc = 0.717371
Epoch 237
Validation auc = 0.715023
Epoch 238
Validation auc = 0.715493
Epoch 239
Loss = 2.1002e-01, PNorm = 44.2865, GNorm = 2.1728, lr_0 = 6.7200e-04
Validation auc = 0.717840
Epoch 240
Validation auc = 0.725352
Epoch 241
Validation auc = 0.729108
Epoch 242
Validation auc = 0.732864
Epoch 243
Validation auc = 0.736150
Epoch 244
Loss = 2.1420e-01, PNorm = 44.4062, GNorm = 2.2416, lr_0 = 6.6625e-04
Validation auc = 0.736150
Epoch 245
Validation auc = 0.737089
Epoch 246
Validation auc = 0.737089
Epoch 247
Validation auc = 0.743662
Epoch 248
Validation auc = 0.743662
Epoch 249
Loss = 2.1093e-01, PNorm = 44.4928, GNorm = 1.9247, lr_0 = 6.6056e-04
Validation auc = 0.741315
Epoch 250
Validation auc = 0.742254
Epoch 251
Validation auc = 0.745070
Epoch 252
Validation auc = 0.739437
Epoch 253
Validation auc = 0.741784
Epoch 254
Loss = 2.4774e-01, PNorm = 44.5642, GNorm = 4.3796, lr_0 = 6.5491e-04
Validation auc = 0.740845
Epoch 255
Validation auc = 0.737559
Epoch 256
Validation auc = 0.732394
Epoch 257
Validation auc = 0.729577
Epoch 258
Validation auc = 0.727230
Epoch 259
Loss = 2.6047e-01, PNorm = 44.6649, GNorm = 2.1701, lr_0 = 6.4931e-04
Validation auc = 0.730516
Epoch 260
Validation auc = 0.736620
Epoch 261
Validation auc = 0.732864
Epoch 262
Validation auc = 0.729577
Epoch 263
Validation auc = 0.727700
Epoch 264
Loss = 1.8792e-01, PNorm = 44.7490, GNorm = 2.5597, lr_0 = 6.4376e-04
Validation auc = 0.728638
Epoch 265
Validation auc = 0.723944
Epoch 266
Validation auc = 0.718310
Epoch 267
Validation auc = 0.711268
Epoch 268
Validation auc = 0.709390
Epoch 269
Loss = 3.5221e-01, PNorm = 44.8222, GNorm = 2.6000, lr_0 = 6.3826e-04
Validation auc = 0.708451
Epoch 270
Validation auc = 0.709859
Epoch 271
Validation auc = 0.710798
Epoch 272
Validation auc = 0.708920
Epoch 273
Validation auc = 0.704225
Epoch 274
Loss = 3.1966e-01, PNorm = 44.9041, GNorm = 4.6139, lr_0 = 6.3280e-04
Validation auc = 0.707512
Epoch 275
Validation auc = 0.706103
Epoch 276
Validation auc = 0.708451
Epoch 277
Validation auc = 0.710798
Epoch 278
Validation auc = 0.716432
Epoch 279
Loss = 2.2695e-01, PNorm = 44.9770, GNorm = 1.6474, lr_0 = 6.2739e-04
Validation auc = 0.720188
Epoch 280
Validation auc = 0.722066
Epoch 281
Validation auc = 0.727230
Epoch 282
Validation auc = 0.731925
Epoch 283
Validation auc = 0.739437
Epoch 284
Loss = 2.6159e-01, PNorm = 45.0653, GNorm = 3.8780, lr_0 = 6.2203e-04
Validation auc = 0.744131
Epoch 285
Validation auc = 0.748357
Epoch 286
Validation auc = 0.747887
Epoch 287
Validation auc = 0.743662
Epoch 288
Validation auc = 0.739437
Epoch 289
Loss = 2.6213e-01, PNorm = 45.1488, GNorm = 2.3032, lr_0 = 6.1671e-04
Validation auc = 0.741315
Epoch 290
Validation auc = 0.737559
Epoch 291
Validation auc = 0.738028
Epoch 292
Validation auc = 0.736620
Epoch 293
Validation auc = 0.733803
Epoch 294
Loss = 1.8566e-01, PNorm = 45.2323, GNorm = 3.1061, lr_0 = 6.1144e-04
Validation auc = 0.732394
Epoch 295
Validation auc = 0.726291
Epoch 296
Validation auc = 0.720188
Epoch 297
Validation auc = 0.723944
Epoch 298
Validation auc = 0.727700
Epoch 299
Loss = 1.8340e-01, PNorm = 45.3164, GNorm = 2.9613, lr_0 = 6.0621e-04
Validation auc = 0.732394
Model 0 best validation auc = 0.751643 on epoch 181
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.918724
Ensemble test auc = 0.918724
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.603286
Epoch 1
Validation auc = 0.682629
Epoch 2
Validation auc = 0.666667
Epoch 3
Validation auc = 0.664789
Epoch 4
Loss = 1.1578e+00, PNorm = 40.6675, GNorm = 18.6631, lr_0 = 6.5000e-04
Validation auc = 0.663850
Epoch 5
Validation auc = 0.662911
Epoch 6
Validation auc = 0.654930
Epoch 7
Validation auc = 0.643192
Epoch 8
Validation auc = 0.621127
Epoch 9
Loss = 8.0703e-01, PNorm = 40.7168, GNorm = 6.9227, lr_0 = 9.9743e-04
Validation auc = 0.627700
Epoch 10
Validation auc = 0.630986
Epoch 11
Validation auc = 0.641315
Epoch 12
Validation auc = 0.638967
Epoch 13
Validation auc = 0.645070
Epoch 14
Loss = 6.0281e-01, PNorm = 40.7580, GNorm = 7.3325, lr_0 = 9.8890e-04
Validation auc = 0.664789
Epoch 15
Validation auc = 0.675117
Epoch 16
Validation auc = 0.676526
Epoch 17
Validation auc = 0.675587
Epoch 18
Validation auc = 0.676526
Epoch 19
Loss = 6.3585e-01, PNorm = 40.8023, GNorm = 6.2515, lr_0 = 9.8045e-04
Validation auc = 0.665258
Epoch 20
Validation auc = 0.649296
Epoch 21
Validation auc = 0.661033
Epoch 22
Validation auc = 0.667606
Epoch 23
Validation auc = 0.688263
Epoch 24
Loss = 7.1434e-01, PNorm = 40.8367, GNorm = 5.4259, lr_0 = 9.7207e-04
Validation auc = 0.714085
Epoch 25
Validation auc = 0.705634
Epoch 26
Validation auc = 0.686854
Epoch 27
Validation auc = 0.686854
Epoch 28
Validation auc = 0.686385
Epoch 29
Loss = 6.4502e-01, PNorm = 40.8571, GNorm = 5.1571, lr_0 = 9.6376e-04
Validation auc = 0.682160
Epoch 30
Validation auc = 0.680282
Epoch 31
Validation auc = 0.690610
Epoch 32
Validation auc = 0.682160
Epoch 33
Validation auc = 0.674648
Epoch 34
Loss = 6.6455e-01, PNorm = 40.8930, GNorm = 5.7153, lr_0 = 9.5552e-04
Validation auc = 0.677934
Epoch 35
Validation auc = 0.678404
Epoch 36
Validation auc = 0.669953
Epoch 37
Validation auc = 0.661502
Epoch 38
Validation auc = 0.668075
Epoch 39
Loss = 6.6550e-01, PNorm = 40.9279, GNorm = 4.4305, lr_0 = 9.4735e-04
Validation auc = 0.684977
Epoch 40
Validation auc = 0.689671
Epoch 41
Validation auc = 0.688732
Epoch 42
Validation auc = 0.680282
Epoch 43
Validation auc = 0.679343
Epoch 44
Loss = 5.7585e-01, PNorm = 40.9649, GNorm = 4.5449, lr_0 = 9.3925e-04
Validation auc = 0.685915
Epoch 45
Validation auc = 0.683568
Epoch 46
Validation auc = 0.692958
Epoch 47
Validation auc = 0.697653
Epoch 48
Validation auc = 0.692488
Epoch 49
Loss = 5.6212e-01, PNorm = 41.0062, GNorm = 3.7860, lr_0 = 9.3122e-04
Validation auc = 0.689202
Epoch 50
Validation auc = 0.680751
Epoch 51
Validation auc = 0.679343
Epoch 52
Validation auc = 0.682160
Epoch 53
Validation auc = 0.686854
Epoch 54
Loss = 6.0842e-01, PNorm = 41.0402, GNorm = 4.9117, lr_0 = 9.2326e-04
Validation auc = 0.683568
Epoch 55
Validation auc = 0.670892
Epoch 56
Validation auc = 0.656808
Epoch 57
Validation auc = 0.656338
Epoch 58
Validation auc = 0.648826
Epoch 59
Loss = 5.3344e-01, PNorm = 41.0928, GNorm = 2.5048, lr_0 = 9.1537e-04
Validation auc = 0.653521
Epoch 60
Validation auc = 0.644601
Epoch 61
Validation auc = 0.646009
Epoch 62
Validation auc = 0.648826
Epoch 63
Validation auc = 0.648826
Epoch 64
Loss = 5.5027e-01, PNorm = 41.1510, GNorm = 2.9258, lr_0 = 9.0754e-04
Validation auc = 0.646479
Epoch 65
Validation auc = 0.638967
Epoch 66
Validation auc = 0.647418
Epoch 67
Validation auc = 0.647887
Epoch 68
Validation auc = 0.647887
Epoch 69
Loss = 5.2007e-01, PNorm = 41.2037, GNorm = 2.7923, lr_0 = 8.9978e-04
Validation auc = 0.641784
Epoch 70
Validation auc = 0.634272
Epoch 71
Validation auc = 0.629577
Epoch 72
Validation auc = 0.631925
Epoch 73
Validation auc = 0.637089
Epoch 74
Loss = 5.7075e-01, PNorm = 41.2577, GNorm = 3.4166, lr_0 = 8.9209e-04
Validation auc = 0.640845
Epoch 75
Validation auc = 0.640376
Epoch 76
Validation auc = 0.639437
Epoch 77
Validation auc = 0.640376
Epoch 78
Validation auc = 0.642254
Epoch 79
Loss = 5.1405e-01, PNorm = 41.3365, GNorm = 3.2384, lr_0 = 8.8447e-04
Validation auc = 0.640845
Epoch 80
Validation auc = 0.633803
Epoch 81
Validation auc = 0.636150
Epoch 82
Validation auc = 0.639437
Epoch 83
Validation auc = 0.647887
Epoch 84
Loss = 4.8568e-01, PNorm = 41.3885, GNorm = 2.7124, lr_0 = 8.7691e-04
Validation auc = 0.653521
Epoch 85
Validation auc = 0.655399
Epoch 86
Validation auc = 0.655399
Epoch 87
Validation auc = 0.655399
Epoch 88
Validation auc = 0.658216
Epoch 89
Loss = 4.8943e-01, PNorm = 41.4778, GNorm = 2.8749, lr_0 = 8.6941e-04
Validation auc = 0.671831
Epoch 90
Validation auc = 0.673709
Epoch 91
Validation auc = 0.652113
Epoch 92
Validation auc = 0.648826
Epoch 93
Validation auc = 0.640376
Epoch 94
Loss = 5.6406e-01, PNorm = 41.5380, GNorm = 2.6476, lr_0 = 8.6198e-04
Validation auc = 0.642254
Epoch 95
Validation auc = 0.649765
Epoch 96
Validation auc = 0.660563
Epoch 97
Validation auc = 0.656808
Epoch 98
Validation auc = 0.656338
Epoch 99
Loss = 5.2838e-01, PNorm = 41.6108, GNorm = 3.2421, lr_0 = 8.5461e-04
Validation auc = 0.672770
Epoch 100
Validation auc = 0.680751
Epoch 101
Validation auc = 0.678873
Epoch 102
Validation auc = 0.682160
Epoch 103
Validation auc = 0.683568
Epoch 104
Loss = 5.5900e-01, PNorm = 41.6980, GNorm = 3.9225, lr_0 = 8.4730e-04
Validation auc = 0.685915
Epoch 105
Validation auc = 0.686385
Epoch 106
Validation auc = 0.690141
Epoch 107
Validation auc = 0.708451
Epoch 108
Validation auc = 0.730516
Epoch 109
Loss = 4.6490e-01, PNorm = 41.7674, GNorm = 3.6123, lr_0 = 8.4006e-04
Validation auc = 0.726291
Epoch 110
Validation auc = 0.697183
Epoch 111
Validation auc = 0.676995
Epoch 112
Validation auc = 0.665728
Epoch 113
Validation auc = 0.670892
Epoch 114
Loss = 5.2028e-01, PNorm = 41.8708, GNorm = 3.4822, lr_0 = 8.3288e-04
Validation auc = 0.687793
Epoch 115
Validation auc = 0.706573
Epoch 116
Validation auc = 0.697183
Epoch 117
Validation auc = 0.689671
Epoch 118
Validation auc = 0.688732
Epoch 119
Loss = 4.5549e-01, PNorm = 41.9655, GNorm = 2.7803, lr_0 = 8.2576e-04
Validation auc = 0.684038
Epoch 120
Validation auc = 0.670892
Epoch 121
Validation auc = 0.671831
Epoch 122
Validation auc = 0.669014
Epoch 123
Validation auc = 0.674178
Epoch 124
Loss = 4.3298e-01, PNorm = 42.0772, GNorm = 2.1209, lr_0 = 8.1870e-04
Validation auc = 0.679343
Epoch 125
Validation auc = 0.677465
Epoch 126
Validation auc = 0.681221
Epoch 127
Validation auc = 0.690610
Epoch 128
Validation auc = 0.687793
Epoch 129
Loss = 4.8987e-01, PNorm = 42.1762, GNorm = 4.0184, lr_0 = 8.1170e-04
Validation auc = 0.683099
Epoch 130
Validation auc = 0.683568
Epoch 131
Validation auc = 0.682629
Epoch 132
Validation auc = 0.690610
Epoch 133
Validation auc = 0.696714
Epoch 134
Loss = 3.9693e-01, PNorm = 42.2874, GNorm = 3.0544, lr_0 = 8.0476e-04
Validation auc = 0.698122
Epoch 135
Validation auc = 0.699531
Epoch 136
Validation auc = 0.699531
Epoch 137
Validation auc = 0.702347
Epoch 138
Validation auc = 0.707981
Epoch 139
Loss = 3.9907e-01, PNorm = 42.4010, GNorm = 2.3495, lr_0 = 7.9788e-04
Validation auc = 0.710329
Epoch 140
Validation auc = 0.719249
Epoch 141
Validation auc = 0.727230
Epoch 142
Validation auc = 0.718310
Epoch 143
Validation auc = 0.711268
Epoch 144
Loss = 4.4270e-01, PNorm = 42.5147, GNorm = 2.9390, lr_0 = 7.9106e-04
Validation auc = 0.701878
Epoch 145
Validation auc = 0.697653
Epoch 146
Validation auc = 0.698592
Epoch 147
Validation auc = 0.707512
Epoch 148
Validation auc = 0.711737
Epoch 149
Loss = 5.1841e-01, PNorm = 42.6306, GNorm = 3.3875, lr_0 = 7.8430e-04
Validation auc = 0.715962
Epoch 150
Validation auc = 0.716901
Epoch 151
Validation auc = 0.715962
Epoch 152
Validation auc = 0.716901
Epoch 153
Validation auc = 0.723474
Epoch 154
Loss = 3.8656e-01, PNorm = 42.7469, GNorm = 3.3456, lr_0 = 7.7759e-04
Validation auc = 0.723944
Epoch 155
Validation auc = 0.722066
Epoch 156
Validation auc = 0.725822
Epoch 157
Validation auc = 0.722535
Epoch 158
Validation auc = 0.724413
Epoch 159
Loss = 3.2216e-01, PNorm = 42.8652, GNorm = 2.8300, lr_0 = 7.7095e-04
Validation auc = 0.723005
Epoch 160
Validation auc = 0.719718
Epoch 161
Validation auc = 0.721596
Epoch 162
Validation auc = 0.719718
Epoch 163
Validation auc = 0.715493
Epoch 164
Loss = 3.6263e-01, PNorm = 42.9752, GNorm = 2.0726, lr_0 = 7.6436e-04
Validation auc = 0.717840
Epoch 165
Validation auc = 0.711737
Epoch 166
Validation auc = 0.710329
Epoch 167
Validation auc = 0.706103
Epoch 168
Validation auc = 0.710798
Epoch 169
Loss = 3.3312e-01, PNorm = 43.0876, GNorm = 1.4397, lr_0 = 7.5782e-04
Validation auc = 0.712207
Epoch 170
Validation auc = 0.710798
Epoch 171
Validation auc = 0.705164
Epoch 172
Validation auc = 0.702347
Epoch 173
Validation auc = 0.702817
Epoch 174
Loss = 3.9319e-01, PNorm = 43.1969, GNorm = 2.2667, lr_0 = 7.5134e-04
Validation auc = 0.707042
Epoch 175
Validation auc = 0.705634
Epoch 176
Validation auc = 0.709390
Epoch 177
Validation auc = 0.711268
Epoch 178
Validation auc = 0.712676
Epoch 179
Loss = 3.5268e-01, PNorm = 43.3053, GNorm = 2.1145, lr_0 = 7.4492e-04
Validation auc = 0.719718
Epoch 180
Validation auc = 0.720188
Epoch 181
Validation auc = 0.722535
Epoch 182
Validation auc = 0.719249
Epoch 183
Validation auc = 0.722535
Epoch 184
Loss = 4.0613e-01, PNorm = 43.4103, GNorm = 3.4378, lr_0 = 7.3855e-04
Validation auc = 0.726291
Epoch 185
Validation auc = 0.722535
Epoch 186
Validation auc = 0.717371
Epoch 187
Validation auc = 0.709859
Epoch 188
Validation auc = 0.707981
Epoch 189
Loss = 3.5514e-01, PNorm = 43.5181, GNorm = 2.6670, lr_0 = 7.3224e-04
Validation auc = 0.714085
Epoch 190
Validation auc = 0.719718
Epoch 191
Validation auc = 0.722535
Epoch 192
Validation auc = 0.735681
Epoch 193
Validation auc = 0.734272
Epoch 194
Loss = 3.9230e-01, PNorm = 43.6279, GNorm = 2.4198, lr_0 = 7.2598e-04
Validation auc = 0.724883
Epoch 195
Validation auc = 0.717371
Epoch 196
Validation auc = 0.715023
Epoch 197
Validation auc = 0.708451
Epoch 198
Validation auc = 0.707042
Epoch 199
Loss = 3.3719e-01, PNorm = 43.7318, GNorm = 1.8758, lr_0 = 7.1977e-04
Validation auc = 0.710329
Epoch 200
Validation auc = 0.706573
Epoch 201
Validation auc = 0.705164
Epoch 202
Validation auc = 0.708451
Epoch 203
Validation auc = 0.718310
Epoch 204
Loss = 3.5699e-01, PNorm = 43.8292, GNorm = 3.3207, lr_0 = 7.1362e-04
Validation auc = 0.726761
Epoch 205
Validation auc = 0.726761
Epoch 206
Validation auc = 0.726291
Epoch 207
Validation auc = 0.726761
Epoch 208
Validation auc = 0.724413
Epoch 209
Loss = 2.8859e-01, PNorm = 43.9209, GNorm = 1.9776, lr_0 = 7.0752e-04
Validation auc = 0.723474
Epoch 210
Validation auc = 0.718779
Epoch 211
Validation auc = 0.719718
Epoch 212
Validation auc = 0.725822
Epoch 213
Validation auc = 0.730986
Epoch 214
Loss = 3.6578e-01, PNorm = 44.0188, GNorm = 3.5555, lr_0 = 7.0147e-04
Validation auc = 0.726291
Epoch 215
Validation auc = 0.718779
Epoch 216
Validation auc = 0.721596
Epoch 217
Validation auc = 0.720657
Epoch 218
Validation auc = 0.723944
Epoch 219
Loss = 3.2868e-01, PNorm = 44.1161, GNorm = 2.0936, lr_0 = 6.9548e-04
Validation auc = 0.723474
Epoch 220
Validation auc = 0.724883
Epoch 221
Validation auc = 0.725822
Epoch 222
Validation auc = 0.728638
Epoch 223
Validation auc = 0.731455
Epoch 224
Loss = 4.2067e-01, PNorm = 44.2279, GNorm = 3.6196, lr_0 = 6.8953e-04
Validation auc = 0.725352
Epoch 225
Validation auc = 0.723944
Epoch 226
Validation auc = 0.725822
Epoch 227
Validation auc = 0.727700
Epoch 228
Validation auc = 0.730047
Epoch 229
Loss = 3.5825e-01, PNorm = 44.3240, GNorm = 3.9799, lr_0 = 6.8364e-04
Validation auc = 0.735211
Epoch 230
Validation auc = 0.732864
Epoch 231
Validation auc = 0.732864
Epoch 232
Validation auc = 0.733803
Epoch 233
Validation auc = 0.739437
Epoch 234
Loss = 3.2741e-01, PNorm = 44.4289, GNorm = 1.8655, lr_0 = 6.7779e-04
Validation auc = 0.738967
Epoch 235
Validation auc = 0.739437
Epoch 236
Validation auc = 0.737559
Epoch 237
Validation auc = 0.734742
Epoch 238
Validation auc = 0.737089
Epoch 239
Loss = 3.4168e-01, PNorm = 44.5322, GNorm = 2.8227, lr_0 = 6.7200e-04
Validation auc = 0.734742
Epoch 240
Validation auc = 0.738028
Epoch 241
Validation auc = 0.738498
Epoch 242
Validation auc = 0.737089
Epoch 243
Validation auc = 0.733803
Epoch 244
Loss = 2.7933e-01, PNorm = 44.6158, GNorm = 2.6784, lr_0 = 6.6625e-04
Validation auc = 0.731455
Epoch 245
Validation auc = 0.732394
Epoch 246
Validation auc = 0.728638
Epoch 247
Validation auc = 0.728638
Epoch 248
Validation auc = 0.727230
Epoch 249
Loss = 3.2775e-01, PNorm = 44.6892, GNorm = 7.4501, lr_0 = 6.6056e-04
Validation auc = 0.729108
Epoch 250
Validation auc = 0.729108
Epoch 251
Validation auc = 0.730986
Epoch 252
Validation auc = 0.729577
Epoch 253
Validation auc = 0.730986
Epoch 254
Loss = 2.8893e-01, PNorm = 44.7835, GNorm = 3.2858, lr_0 = 6.5491e-04
Validation auc = 0.731925
Epoch 255
Validation auc = 0.732864
Epoch 256
Validation auc = 0.735681
Epoch 257
Validation auc = 0.735211
Epoch 258
Validation auc = 0.736620
Epoch 259
Loss = 3.0361e-01, PNorm = 44.8716, GNorm = 2.5285, lr_0 = 6.4931e-04
Validation auc = 0.739906
Epoch 260
Validation auc = 0.741784
Epoch 261
Validation auc = 0.738028
Epoch 262
Validation auc = 0.738498
Epoch 263
Validation auc = 0.739906
Epoch 264
Loss = 3.1136e-01, PNorm = 44.9499, GNorm = 2.4712, lr_0 = 6.4376e-04
Validation auc = 0.746948
Epoch 265
Validation auc = 0.749296
Epoch 266
Validation auc = 0.744601
Epoch 267
Validation auc = 0.744131
Epoch 268
Validation auc = 0.736620
Epoch 269
Loss = 2.5056e-01, PNorm = 45.0418, GNorm = 3.1355, lr_0 = 6.3826e-04
Validation auc = 0.735211
Epoch 270
Validation auc = 0.723944
Epoch 271
Validation auc = 0.719718
Epoch 272
Validation auc = 0.721127
Epoch 273
Validation auc = 0.722066
Epoch 274
Loss = 2.6467e-01, PNorm = 45.1052, GNorm = 2.6963, lr_0 = 6.3280e-04
Validation auc = 0.726291
Epoch 275
Validation auc = 0.727230
Epoch 276
Validation auc = 0.729577
Epoch 277
Validation auc = 0.725352
Epoch 278
Validation auc = 0.727700
Epoch 279
Loss = 2.8762e-01, PNorm = 45.1775, GNorm = 1.6946, lr_0 = 6.2739e-04
Validation auc = 0.726291
Epoch 280
Validation auc = 0.729577
Epoch 281
Validation auc = 0.731925
Epoch 282
Validation auc = 0.730986
Epoch 283
Validation auc = 0.732394
Epoch 284
Loss = 2.6083e-01, PNorm = 45.2759, GNorm = 2.3677, lr_0 = 6.2203e-04
Validation auc = 0.732864
Epoch 285
Validation auc = 0.739437
Epoch 286
Validation auc = 0.738967
Epoch 287
Validation auc = 0.735681
Epoch 288
Validation auc = 0.723005
Epoch 289
Loss = 1.8965e-01, PNorm = 45.3599, GNorm = 2.6017, lr_0 = 6.1671e-04
Validation auc = 0.715962
Epoch 290
Validation auc = 0.714554
Epoch 291
Validation auc = 0.721596
Epoch 292
Validation auc = 0.726291
Epoch 293
Validation auc = 0.730047
Epoch 294
Loss = 1.6455e-01, PNorm = 45.4562, GNorm = 1.4983, lr_0 = 6.1144e-04
Validation auc = 0.732394
Epoch 295
Validation auc = 0.742254
Epoch 296
Validation auc = 0.745540
Epoch 297
Validation auc = 0.741784
Epoch 298
Validation auc = 0.735681
Epoch 299
Loss = 1.6366e-01, PNorm = 45.5460, GNorm = 1.6271, lr_0 = 6.0621e-04
Validation auc = 0.734742
Model 0 best validation auc = 0.749296 on epoch 265
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.919326
Ensemble test auc = 0.919326
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.536150
Epoch 1
Validation auc = 0.675117
Epoch 2
Validation auc = 0.665728
Epoch 3
Validation auc = 0.670892
Epoch 4
Loss = 1.1533e+00, PNorm = 40.6663, GNorm = 16.0173, lr_0 = 6.5000e-04
Validation auc = 0.692488
Epoch 5
Validation auc = 0.704695
Epoch 6
Validation auc = 0.703756
Epoch 7
Validation auc = 0.704695
Epoch 8
Validation auc = 0.712676
Epoch 9
Loss = 8.8233e-01, PNorm = 40.7134, GNorm = 9.5232, lr_0 = 9.9743e-04
Validation auc = 0.735681
Epoch 10
Validation auc = 0.696244
Epoch 11
Validation auc = 0.669953
Epoch 12
Validation auc = 0.666197
Epoch 13
Validation auc = 0.657746
Epoch 14
Loss = 8.9626e-01, PNorm = 40.7581, GNorm = 7.0457, lr_0 = 9.8890e-04
Validation auc = 0.694836
Epoch 15
Validation auc = 0.670423
Epoch 16
Validation auc = 0.659155
Epoch 17
Validation auc = 0.658685
Epoch 18
Validation auc = 0.651643
Epoch 19
Loss = 7.1981e-01, PNorm = 40.8034, GNorm = 6.7761, lr_0 = 9.8045e-04
Validation auc = 0.634272
Epoch 20
Validation auc = 0.637089
Epoch 21
Validation auc = 0.633333
Epoch 22
Validation auc = 0.642723
Epoch 23
Validation auc = 0.650704
Epoch 24
Loss = 6.1751e-01, PNorm = 40.8444, GNorm = 7.1741, lr_0 = 9.7207e-04
Validation auc = 0.648357
Epoch 25
Validation auc = 0.661972
Epoch 26
Validation auc = 0.672300
Epoch 27
Validation auc = 0.684507
Epoch 28
Validation auc = 0.700469
Epoch 29
Loss = 5.6580e-01, PNorm = 40.8728, GNorm = 4.9620, lr_0 = 9.6376e-04
Validation auc = 0.667136
Epoch 30
Validation auc = 0.663850
Epoch 31
Validation auc = 0.666197
Epoch 32
Validation auc = 0.674648
Epoch 33
Validation auc = 0.680751
Epoch 34
Loss = 7.1216e-01, PNorm = 40.9066, GNorm = 6.3080, lr_0 = 9.5552e-04
Validation auc = 0.692958
Epoch 35
Validation auc = 0.689202
Epoch 36
Validation auc = 0.685915
Epoch 37
Validation auc = 0.680282
Epoch 38
Validation auc = 0.677465
Epoch 39
Loss = 6.6779e-01, PNorm = 40.9509, GNorm = 4.4193, lr_0 = 9.4735e-04
Validation auc = 0.665728
Epoch 40
Validation auc = 0.642723
Epoch 41
Validation auc = 0.618779
Epoch 42
Validation auc = 0.630516
Epoch 43
Validation auc = 0.644131
Epoch 44
Loss = 5.5403e-01, PNorm = 40.9773, GNorm = 5.3221, lr_0 = 9.3925e-04
Validation auc = 0.642254
Epoch 45
Validation auc = 0.637559
Epoch 46
Validation auc = 0.624413
Epoch 47
Validation auc = 0.667606
Epoch 48
Validation auc = 0.682629
Epoch 49
Loss = 5.5389e-01, PNorm = 41.0108, GNorm = 3.2024, lr_0 = 9.3122e-04
Validation auc = 0.669014
Epoch 50
Validation auc = 0.653521
Epoch 51
Validation auc = 0.644131
Epoch 52
Validation auc = 0.646009
Epoch 53
Validation auc = 0.654460
Epoch 54
Loss = 5.4156e-01, PNorm = 41.0342, GNorm = 3.5037, lr_0 = 9.2326e-04
Validation auc = 0.653521
Epoch 55
Validation auc = 0.655869
Epoch 56
Validation auc = 0.650235
Epoch 57
Validation auc = 0.655869
Epoch 58
Validation auc = 0.652113
Epoch 59
Loss = 5.5974e-01, PNorm = 41.0695, GNorm = 4.2071, lr_0 = 9.1537e-04
Validation auc = 0.646009
Epoch 60
Validation auc = 0.644601
Epoch 61
Validation auc = 0.639437
Epoch 62
Validation auc = 0.637089
Epoch 63
Validation auc = 0.649765
Epoch 64
Loss = 5.7852e-01, PNorm = 41.1212, GNorm = 3.7879, lr_0 = 9.0754e-04
Validation auc = 0.659155
Epoch 65
Validation auc = 0.659624
Epoch 66
Validation auc = 0.657746
Epoch 67
Validation auc = 0.657746
Epoch 68
Validation auc = 0.655399
Epoch 69
Loss = 5.3268e-01, PNorm = 41.1519, GNorm = 3.4951, lr_0 = 8.9978e-04
Validation auc = 0.644601
Epoch 70
Validation auc = 0.642254
Epoch 71
Validation auc = 0.644131
Epoch 72
Validation auc = 0.649296
Epoch 73
Validation auc = 0.650235
Epoch 74
Loss = 6.3504e-01, PNorm = 41.2065, GNorm = 2.7114, lr_0 = 8.9209e-04
Validation auc = 0.635211
Epoch 75
Validation auc = 0.622535
Epoch 76
Validation auc = 0.628638
Epoch 77
Validation auc = 0.639437
Epoch 78
Validation auc = 0.660563
Epoch 79
Loss = 5.4777e-01, PNorm = 41.2713, GNorm = 3.4111, lr_0 = 8.8447e-04
Validation auc = 0.665258
Epoch 80
Validation auc = 0.670423
Epoch 81
Validation auc = 0.666667
Epoch 82
Validation auc = 0.655869
Epoch 83
Validation auc = 0.649765
Epoch 84
Loss = 5.1738e-01, PNorm = 41.3113, GNorm = 2.7891, lr_0 = 8.7691e-04
Validation auc = 0.638028
Epoch 85
Validation auc = 0.635211
Epoch 86
Validation auc = 0.636620
Epoch 87
Validation auc = 0.639437
Epoch 88
Validation auc = 0.643662
Epoch 89
Loss = 4.7493e-01, PNorm = 41.3699, GNorm = 2.6341, lr_0 = 8.6941e-04
Validation auc = 0.647887
Epoch 90
Validation auc = 0.648826
Epoch 91
Validation auc = 0.666197
Epoch 92
Validation auc = 0.674648
Epoch 93
Validation auc = 0.677465
Epoch 94
Loss = 5.5329e-01, PNorm = 41.4377, GNorm = 3.8990, lr_0 = 8.6198e-04
Validation auc = 0.681221
Epoch 95
Validation auc = 0.692958
Epoch 96
Validation auc = 0.697183
Epoch 97
Validation auc = 0.686854
Epoch 98
Validation auc = 0.667136
Epoch 99
Loss = 5.0480e-01, PNorm = 41.5209, GNorm = 2.3855, lr_0 = 8.5461e-04
Validation auc = 0.642723
Epoch 100
Validation auc = 0.646948
Epoch 101
Validation auc = 0.646009
Epoch 102
Validation auc = 0.656338
Epoch 103
Validation auc = 0.666667
Epoch 104
Loss = 4.5554e-01, PNorm = 41.5956, GNorm = 1.9447, lr_0 = 8.4730e-04
Validation auc = 0.669014
Epoch 105
Validation auc = 0.661502
Epoch 106
Validation auc = 0.657277
Epoch 107
Validation auc = 0.661972
Epoch 108
Validation auc = 0.671362
Epoch 109
Loss = 4.9870e-01, PNorm = 41.6633, GNorm = 3.1835, lr_0 = 8.4006e-04
Validation auc = 0.678404
Epoch 110
Validation auc = 0.680282
Epoch 111
Validation auc = 0.680751
Epoch 112
Validation auc = 0.681221
Epoch 113
Validation auc = 0.672300
Epoch 114
Loss = 5.2354e-01, PNorm = 41.7399, GNorm = 2.3512, lr_0 = 8.3288e-04
Validation auc = 0.667606
Epoch 115
Validation auc = 0.662911
Epoch 116
Validation auc = 0.654460
Epoch 117
Validation auc = 0.657277
Epoch 118
Validation auc = 0.672300
Epoch 119
Loss = 5.0476e-01, PNorm = 41.8358, GNorm = 3.2639, lr_0 = 8.2576e-04
Validation auc = 0.693897
Epoch 120
Validation auc = 0.697183
Epoch 121
Validation auc = 0.700939
Epoch 122
Validation auc = 0.680282
Epoch 123
Validation auc = 0.678873
Epoch 124
Loss = 4.0270e-01, PNorm = 41.9268, GNorm = 2.1858, lr_0 = 8.1870e-04
Validation auc = 0.679812
Epoch 125
Validation auc = 0.682160
Epoch 126
Validation auc = 0.687793
Epoch 127
Validation auc = 0.697653
Epoch 128
Validation auc = 0.692488
Epoch 129
Loss = 4.5132e-01, PNorm = 42.0280, GNorm = 3.1844, lr_0 = 8.1170e-04
Validation auc = 0.691080
Epoch 130
Validation auc = 0.687793
Epoch 131
Validation auc = 0.692488
Epoch 132
Validation auc = 0.707042
Epoch 133
Validation auc = 0.698592
Epoch 134
Loss = 4.5658e-01, PNorm = 42.1328, GNorm = 3.6524, lr_0 = 8.0476e-04
Validation auc = 0.701878
Epoch 135
Validation auc = 0.692958
Epoch 136
Validation auc = 0.704695
Epoch 137
Validation auc = 0.706103
Epoch 138
Validation auc = 0.707042
Epoch 139
Loss = 4.8805e-01, PNorm = 42.2145, GNorm = 3.3067, lr_0 = 7.9788e-04
Validation auc = 0.709859
Epoch 140
Validation auc = 0.715962
Epoch 141
Validation auc = 0.733333
Epoch 142
Validation auc = 0.737559
Epoch 143
Validation auc = 0.734742
Epoch 144
Loss = 4.4108e-01, PNorm = 42.3014, GNorm = 3.2396, lr_0 = 7.9106e-04
Validation auc = 0.730047
Epoch 145
Validation auc = 0.731455
Epoch 146
Validation auc = 0.723005
Epoch 147
Validation auc = 0.730986
Epoch 148
Validation auc = 0.737559
Epoch 149
Loss = 4.3145e-01, PNorm = 42.4123, GNorm = 3.2729, lr_0 = 7.8430e-04
Validation auc = 0.745070
Epoch 150
Validation auc = 0.763850
Epoch 151
Validation auc = 0.756338
Epoch 152
Validation auc = 0.752113
Epoch 153
Validation auc = 0.743192
Epoch 154
Loss = 4.0937e-01, PNorm = 42.4988, GNorm = 4.3696, lr_0 = 7.7759e-04
Validation auc = 0.743192
Epoch 155
Validation auc = 0.743192
Epoch 156
Validation auc = 0.722535
Epoch 157
Validation auc = 0.707512
Epoch 158
Validation auc = 0.696714
Epoch 159
Loss = 4.1342e-01, PNorm = 42.5807, GNorm = 3.4529, lr_0 = 7.7095e-04
Validation auc = 0.691549
Epoch 160
Validation auc = 0.700469
Epoch 161
Validation auc = 0.710798
Epoch 162
Validation auc = 0.738028
Epoch 163
Validation auc = 0.736150
Epoch 164
Loss = 3.7821e-01, PNorm = 42.6576, GNorm = 2.0713, lr_0 = 7.6436e-04
Validation auc = 0.742723
Epoch 165
Validation auc = 0.738967
Epoch 166
Validation auc = 0.733333
Epoch 167
Validation auc = 0.724413
Epoch 168
Validation auc = 0.722066
Epoch 169
Loss = 3.7799e-01, PNorm = 42.7566, GNorm = 2.0476, lr_0 = 7.5782e-04
Validation auc = 0.722066
Epoch 170
Validation auc = 0.720188
Epoch 171
Validation auc = 0.728638
Epoch 172
Validation auc = 0.729108
Epoch 173
Validation auc = 0.735681
Epoch 174
Loss = 3.8103e-01, PNorm = 42.8446, GNorm = 2.5770, lr_0 = 7.5134e-04
Validation auc = 0.727700
Epoch 175
Validation auc = 0.733333
Epoch 176
Validation auc = 0.730516
Epoch 177
Validation auc = 0.729108
Epoch 178
Validation auc = 0.734742
Epoch 179
Loss = 4.0257e-01, PNorm = 42.9218, GNorm = 2.5655, lr_0 = 7.4492e-04
Validation auc = 0.750235
Epoch 180
Validation auc = 0.741315
Epoch 181
Validation auc = 0.725352
Epoch 182
Validation auc = 0.710798
Epoch 183
Validation auc = 0.714554
Epoch 184
Loss = 3.5298e-01, PNorm = 42.9909, GNorm = 1.9254, lr_0 = 7.3855e-04
Validation auc = 0.726761
Epoch 185
Validation auc = 0.730516
Epoch 186
Validation auc = 0.729577
Epoch 187
Validation auc = 0.715493
Epoch 188
Validation auc = 0.702817
Epoch 189
Loss = 3.3940e-01, PNorm = 43.0687, GNorm = 2.4999, lr_0 = 7.3224e-04
Validation auc = 0.690610
Epoch 190
Validation auc = 0.695305
Epoch 191
Validation auc = 0.707512
Epoch 192
Validation auc = 0.710329
Epoch 193
Validation auc = 0.717371
Epoch 194
Loss = 3.5784e-01, PNorm = 43.1454, GNorm = 4.3917, lr_0 = 7.2598e-04
Validation auc = 0.717371
Epoch 195
Validation auc = 0.718310
Epoch 196
Validation auc = 0.719718
Epoch 197
Validation auc = 0.733333
Epoch 198
Validation auc = 0.730986
Epoch 199
Loss = 3.5423e-01, PNorm = 43.2422, GNorm = 2.2942, lr_0 = 7.1977e-04
Validation auc = 0.715962
Epoch 200
Validation auc = 0.706103
Epoch 201
Validation auc = 0.705164
Epoch 202
Validation auc = 0.716432
Epoch 203
Validation auc = 0.723474
Epoch 204
Loss = 3.6390e-01, PNorm = 43.3271, GNorm = 2.4569, lr_0 = 7.1362e-04
Validation auc = 0.711737
Epoch 205
Validation auc = 0.711268
Epoch 206
Validation auc = 0.715023
Epoch 207
Validation auc = 0.719249
Epoch 208
Validation auc = 0.723474
Epoch 209
Loss = 2.7273e-01, PNorm = 43.4115, GNorm = 3.6572, lr_0 = 7.0752e-04
Validation auc = 0.723474
Epoch 210
Validation auc = 0.719249
Epoch 211
Validation auc = 0.723005
Epoch 212
Validation auc = 0.734272
Epoch 213
Validation auc = 0.731455
Epoch 214
Loss = 3.7233e-01, PNorm = 43.4889, GNorm = 2.5126, lr_0 = 7.0147e-04
Validation auc = 0.721596
Epoch 215
Validation auc = 0.714085
Epoch 216
Validation auc = 0.711737
Epoch 217
Validation auc = 0.716901
Epoch 218
Validation auc = 0.724883
Epoch 219
Loss = 3.0311e-01, PNorm = 43.5604, GNorm = 3.0114, lr_0 = 6.9548e-04
Validation auc = 0.730516
Epoch 220
Validation auc = 0.729577
Epoch 221
Validation auc = 0.733333
Epoch 222
Validation auc = 0.735681
Epoch 223
Validation auc = 0.735681
Epoch 224
Loss = 3.3344e-01, PNorm = 43.6485, GNorm = 2.1899, lr_0 = 6.8953e-04
Validation auc = 0.737559
Epoch 225
Validation auc = 0.743192
Epoch 226
Validation auc = 0.749296
Epoch 227
Validation auc = 0.750235
Epoch 228
Validation auc = 0.754460
Epoch 229
Loss = 3.1298e-01, PNorm = 43.7215, GNorm = 2.8020, lr_0 = 6.8364e-04
Validation auc = 0.744601
Epoch 230
Validation auc = 0.742723
Epoch 231
Validation auc = 0.741315
Epoch 232
Validation auc = 0.740845
Epoch 233
Validation auc = 0.743192
Epoch 234
Loss = 2.9451e-01, PNorm = 43.8068, GNorm = 2.1586, lr_0 = 6.7779e-04
Validation auc = 0.748826
Epoch 235
Validation auc = 0.752582
Epoch 236
Validation auc = 0.754930
Epoch 237
Validation auc = 0.757277
Epoch 238
Validation auc = 0.756338
Epoch 239
Loss = 3.2507e-01, PNorm = 43.8931, GNorm = 2.4427, lr_0 = 6.7200e-04
Validation auc = 0.752582
Epoch 240
Validation auc = 0.741784
Epoch 241
Validation auc = 0.730047
Epoch 242
Validation auc = 0.742723
Epoch 243
Validation auc = 0.737559
Epoch 244
Loss = 2.8611e-01, PNorm = 43.9835, GNorm = 2.9700, lr_0 = 6.6625e-04
Validation auc = 0.728638
Epoch 245
Validation auc = 0.722535
Epoch 246
Validation auc = 0.722535
Epoch 247
Validation auc = 0.729108
Epoch 248
Validation auc = 0.733333
Epoch 249
Loss = 2.2796e-01, PNorm = 44.0774, GNorm = 3.1706, lr_0 = 6.6056e-04
Validation auc = 0.724883
Epoch 250
Validation auc = 0.720188
Epoch 251
Validation auc = 0.730986
Epoch 252
Validation auc = 0.731455
Epoch 253
Validation auc = 0.720188
Epoch 254
Loss = 2.7918e-01, PNorm = 44.1538, GNorm = 2.0900, lr_0 = 6.5491e-04
Validation auc = 0.711268
Epoch 255
Validation auc = 0.721596
Epoch 256
Validation auc = 0.727230
Epoch 257
Validation auc = 0.715023
Epoch 258
Validation auc = 0.705164
Epoch 259
Loss = 3.4441e-01, PNorm = 44.2343, GNorm = 2.7034, lr_0 = 6.4931e-04
Validation auc = 0.710329
Epoch 260
Validation auc = 0.716432
Epoch 261
Validation auc = 0.719718
Epoch 262
Validation auc = 0.726291
Epoch 263
Validation auc = 0.728169
Epoch 264
Loss = 2.9932e-01, PNorm = 44.3186, GNorm = 2.1079, lr_0 = 6.4376e-04
Validation auc = 0.723944
Epoch 265
Validation auc = 0.717371
Epoch 266
Validation auc = 0.722066
Epoch 267
Validation auc = 0.724413
Epoch 268
Validation auc = 0.730047
Epoch 269
Loss = 2.8537e-01, PNorm = 44.3823, GNorm = 2.3619, lr_0 = 6.3826e-04
Validation auc = 0.730516
Epoch 270
Validation auc = 0.735211
Epoch 271
Validation auc = 0.735211
Epoch 272
Validation auc = 0.738498
Epoch 273
Validation auc = 0.738967
Epoch 274
Loss = 2.1924e-01, PNorm = 44.4664, GNorm = 1.8456, lr_0 = 6.3280e-04
Validation auc = 0.737089
Epoch 275
Validation auc = 0.735211
Epoch 276
Validation auc = 0.735681
Epoch 277
Validation auc = 0.740376
Epoch 278
Validation auc = 0.746479
Epoch 279
Loss = 2.8099e-01, PNorm = 44.5584, GNorm = 4.6434, lr_0 = 6.2739e-04
Validation auc = 0.745070
Epoch 280
Validation auc = 0.745070
Epoch 281
Validation auc = 0.739437
Epoch 282
Validation auc = 0.732394
Epoch 283
Validation auc = 0.732394
Epoch 284
Loss = 2.1668e-01, PNorm = 44.6429, GNorm = 2.5599, lr_0 = 6.2203e-04
Validation auc = 0.722535
Epoch 285
Validation auc = 0.714085
Epoch 286
Validation auc = 0.708920
Epoch 287
Validation auc = 0.708920
Epoch 288
Validation auc = 0.710798
Epoch 289
Loss = 2.8424e-01, PNorm = 44.7194, GNorm = 1.3332, lr_0 = 6.1671e-04
Validation auc = 0.712207
Epoch 290
Validation auc = 0.715493
Epoch 291
Validation auc = 0.721127
Epoch 292
Validation auc = 0.728638
Epoch 293
Validation auc = 0.733803
Epoch 294
Loss = 2.8917e-01, PNorm = 44.7856, GNorm = 3.2349, lr_0 = 6.1144e-04
Validation auc = 0.733803
Epoch 295
Validation auc = 0.738028
Epoch 296
Validation auc = 0.739437
Epoch 297
Validation auc = 0.739437
Epoch 298
Validation auc = 0.748826
Epoch 299
Loss = 2.6445e-01, PNorm = 44.8719, GNorm = 2.5701, lr_0 = 6.0621e-04
Validation auc = 0.759155
Model 0 best validation auc = 0.763850 on epoch 150
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.913305
Ensemble test auc = 0.913305
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.523944
Epoch 1
Validation auc = 0.680751
Epoch 2
Validation auc = 0.669014
Epoch 3
Validation auc = 0.669484
Epoch 4
Loss = 1.3235e+00, PNorm = 40.6679, GNorm = 10.8348, lr_0 = 6.5000e-04
Validation auc = 0.676056
Epoch 5
Validation auc = 0.673709
Epoch 6
Validation auc = 0.669014
Epoch 7
Validation auc = 0.662441
Epoch 8
Validation auc = 0.657277
Epoch 9
Loss = 9.6073e-01, PNorm = 40.7089, GNorm = 10.5573, lr_0 = 9.9743e-04
Validation auc = 0.638967
Epoch 10
Validation auc = 0.656338
Epoch 11
Validation auc = 0.657277
Epoch 12
Validation auc = 0.656808
Epoch 13
Validation auc = 0.662911
Epoch 14
Loss = 7.9041e-01, PNorm = 40.7595, GNorm = 8.7374, lr_0 = 9.8890e-04
Validation auc = 0.654460
Epoch 15
Validation auc = 0.696714
Epoch 16
Validation auc = 0.658685
Epoch 17
Validation auc = 0.676526
Epoch 18
Validation auc = 0.673239
Epoch 19
Loss = 8.1752e-01, PNorm = 40.8060, GNorm = 8.7018, lr_0 = 9.8045e-04
Validation auc = 0.673239
Epoch 20
Validation auc = 0.690610
Epoch 21
Validation auc = 0.698122
Epoch 22
Validation auc = 0.694836
Epoch 23
Validation auc = 0.692488
Epoch 24
Loss = 5.8658e-01, PNorm = 40.8512, GNorm = 4.6600, lr_0 = 9.7207e-04
Validation auc = 0.696244
Epoch 25
Validation auc = 0.693897
Epoch 26
Validation auc = 0.702817
Epoch 27
Validation auc = 0.697653
Epoch 28
Validation auc = 0.662911
Epoch 29
Loss = 6.6766e-01, PNorm = 40.8789, GNorm = 6.1643, lr_0 = 9.6376e-04
Validation auc = 0.657277
Epoch 30
Validation auc = 0.660094
Epoch 31
Validation auc = 0.653521
Epoch 32
Validation auc = 0.643192
Epoch 33
Validation auc = 0.643662
Epoch 34
Loss = 6.8848e-01, PNorm = 40.9078, GNorm = 5.5601, lr_0 = 9.5552e-04
Validation auc = 0.646948
Epoch 35
Validation auc = 0.674648
Epoch 36
Validation auc = 0.667136
Epoch 37
Validation auc = 0.666667
Epoch 38
Validation auc = 0.664789
Epoch 39
Loss = 5.7763e-01, PNorm = 40.9449, GNorm = 3.8607, lr_0 = 9.4735e-04
Validation auc = 0.680751
Epoch 40
Validation auc = 0.672300
Epoch 41
Validation auc = 0.664319
Epoch 42
Validation auc = 0.663380
Epoch 43
Validation auc = 0.671831
Epoch 44
Loss = 4.8183e-01, PNorm = 40.9914, GNorm = 3.5697, lr_0 = 9.3925e-04
Validation auc = 0.708920
Epoch 45
Validation auc = 0.690141
Epoch 46
Validation auc = 0.697653
Epoch 47
Validation auc = 0.684038
Epoch 48
Validation auc = 0.668545
Epoch 49
Loss = 6.0177e-01, PNorm = 41.0231, GNorm = 3.2645, lr_0 = 9.3122e-04
Validation auc = 0.666667
Epoch 50
Validation auc = 0.664319
Epoch 51
Validation auc = 0.668075
Epoch 52
Validation auc = 0.665258
Epoch 53
Validation auc = 0.665728
Epoch 54
Loss = 5.6454e-01, PNorm = 41.0591, GNorm = 2.8316, lr_0 = 9.2326e-04
Validation auc = 0.663380
Epoch 55
Validation auc = 0.657746
Epoch 56
Validation auc = 0.651643
Epoch 57
Validation auc = 0.662911
Epoch 58
Validation auc = 0.661972
Epoch 59
Loss = 5.8134e-01, PNorm = 41.1075, GNorm = 3.9781, lr_0 = 9.1537e-04
Validation auc = 0.656808
Epoch 60
Validation auc = 0.647418
Epoch 61
Validation auc = 0.644131
Epoch 62
Validation auc = 0.649765
Epoch 63
Validation auc = 0.653052
Epoch 64
Loss = 5.2201e-01, PNorm = 41.1730, GNorm = 3.2161, lr_0 = 9.0754e-04
Validation auc = 0.653521
Epoch 65
Validation auc = 0.656808
Epoch 66
Validation auc = 0.655869
Epoch 67
Validation auc = 0.651643
Epoch 68
Validation auc = 0.646948
Epoch 69
Loss = 5.9220e-01, PNorm = 41.2163, GNorm = 3.6572, lr_0 = 8.9978e-04
Validation auc = 0.649296
Epoch 70
Validation auc = 0.638967
Epoch 71
Validation auc = 0.629108
Epoch 72
Validation auc = 0.632864
Epoch 73
Validation auc = 0.638028
Epoch 74
Loss = 5.5007e-01, PNorm = 41.2861, GNorm = 3.2879, lr_0 = 8.9209e-04
Validation auc = 0.640845
Epoch 75
Validation auc = 0.642254
Epoch 76
Validation auc = 0.640845
Epoch 77
Validation auc = 0.630047
Epoch 78
Validation auc = 0.626291
Epoch 79
Loss = 5.0204e-01, PNorm = 41.3389, GNorm = 3.6914, lr_0 = 8.8447e-04
Validation auc = 0.623474
Epoch 80
Validation auc = 0.619718
Epoch 81
Validation auc = 0.623474
Epoch 82
Validation auc = 0.629577
Epoch 83
Validation auc = 0.636620
Epoch 84
Loss = 4.2371e-01, PNorm = 41.4078, GNorm = 3.2763, lr_0 = 8.7691e-04
Validation auc = 0.642254
Epoch 85
Validation auc = 0.640376
Epoch 86
Validation auc = 0.649296
Epoch 87
Validation auc = 0.655399
Epoch 88
Validation auc = 0.659155
Epoch 89
Loss = 4.7340e-01, PNorm = 41.4651, GNorm = 2.1608, lr_0 = 8.6941e-04
Validation auc = 0.651643
Epoch 90
Validation auc = 0.646009
Epoch 91
Validation auc = 0.656338
Epoch 92
Validation auc = 0.660094
Epoch 93
Validation auc = 0.662911
Epoch 94
Loss = 4.8955e-01, PNorm = 41.5409, GNorm = 2.3912, lr_0 = 8.6198e-04
Validation auc = 0.644131
Epoch 95
Validation auc = 0.646009
Epoch 96
Validation auc = 0.643662
Epoch 97
Validation auc = 0.638498
Epoch 98
Validation auc = 0.638498
Epoch 99
Loss = 4.7300e-01, PNorm = 41.5982, GNorm = 2.8803, lr_0 = 8.5461e-04
Validation auc = 0.634742
Epoch 100
Validation auc = 0.650235
Epoch 101
Validation auc = 0.665258
Epoch 102
Validation auc = 0.668075
Epoch 103
Validation auc = 0.662911
Epoch 104
Loss = 4.9217e-01, PNorm = 41.6819, GNorm = 2.8168, lr_0 = 8.4730e-04
Validation auc = 0.661033
Epoch 105
Validation auc = 0.657277
Epoch 106
Validation auc = 0.643662
Epoch 107
Validation auc = 0.639437
Epoch 108
Validation auc = 0.644601
Epoch 109
Loss = 4.1850e-01, PNorm = 41.7752, GNorm = 2.7337, lr_0 = 8.4006e-04
Validation auc = 0.648357
Epoch 110
Validation auc = 0.657746
Epoch 111
Validation auc = 0.658685
Epoch 112
Validation auc = 0.659155
Epoch 113
Validation auc = 0.652582
Epoch 114
Loss = 4.3138e-01, PNorm = 41.8783, GNorm = 2.6740, lr_0 = 8.3288e-04
Validation auc = 0.652113
Epoch 115
Validation auc = 0.652113
Epoch 116
Validation auc = 0.651643
Epoch 117
Validation auc = 0.653521
Epoch 118
Validation auc = 0.662441
Epoch 119
Loss = 4.1232e-01, PNorm = 41.9849, GNorm = 3.0759, lr_0 = 8.2576e-04
Validation auc = 0.668545
Epoch 120
Validation auc = 0.676526
Epoch 121
Validation auc = 0.668075
Epoch 122
Validation auc = 0.670423
Epoch 123
Validation auc = 0.675117
Epoch 124
Loss = 4.1703e-01, PNorm = 42.0741, GNorm = 2.3914, lr_0 = 8.1870e-04
Validation auc = 0.678404
Epoch 125
Validation auc = 0.675587
Epoch 126
Validation auc = 0.673709
Epoch 127
Validation auc = 0.673709
Epoch 128
Validation auc = 0.684038
Epoch 129
Loss = 4.0811e-01, PNorm = 42.1763, GNorm = 2.8827, lr_0 = 8.1170e-04
Validation auc = 0.680282
Epoch 130
Validation auc = 0.679343
Epoch 131
Validation auc = 0.688263
Epoch 132
Validation auc = 0.690141
Epoch 133
Validation auc = 0.687793
Epoch 134
Loss = 4.5039e-01, PNorm = 42.2607, GNorm = 3.5740, lr_0 = 8.0476e-04
Validation auc = 0.692488
Epoch 135
Validation auc = 0.695775
Epoch 136
Validation auc = 0.694836
Epoch 137
Validation auc = 0.700469
Epoch 138
Validation auc = 0.700939
Epoch 139
Loss = 4.2703e-01, PNorm = 42.3466, GNorm = 2.6820, lr_0 = 7.9788e-04
Validation auc = 0.698592
Epoch 140
Validation auc = 0.694836
Epoch 141
Validation auc = 0.703286
Epoch 142
Validation auc = 0.698592
Epoch 143
Validation auc = 0.694836
Epoch 144
Loss = 4.6522e-01, PNorm = 42.4414, GNorm = 2.5665, lr_0 = 7.9106e-04
Validation auc = 0.699061
Epoch 145
Validation auc = 0.708920
Epoch 146
Validation auc = 0.707512
Epoch 147
Validation auc = 0.712207
Epoch 148
Validation auc = 0.714085
Epoch 149
Loss = 3.5129e-01, PNorm = 42.5515, GNorm = 2.9054, lr_0 = 7.8430e-04
Validation auc = 0.712676
Epoch 150
Validation auc = 0.711737
Epoch 151
Validation auc = 0.712207
Epoch 152
Validation auc = 0.721127
Epoch 153
Validation auc = 0.717840
Epoch 154
Loss = 3.5110e-01, PNorm = 42.6587, GNorm = 2.8926, lr_0 = 7.7759e-04
Validation auc = 0.715023
Epoch 155
Validation auc = 0.715493
Epoch 156
Validation auc = 0.723474
Epoch 157
Validation auc = 0.729577
Epoch 158
Validation auc = 0.725352
Epoch 159
Loss = 3.2225e-01, PNorm = 42.7444, GNorm = 2.4868, lr_0 = 7.7095e-04
Validation auc = 0.713615
Epoch 160
Validation auc = 0.708920
Epoch 161
Validation auc = 0.717840
Epoch 162
Validation auc = 0.719249
Epoch 163
Validation auc = 0.711737
Epoch 164
Loss = 5.0058e-01, PNorm = 42.8353, GNorm = 3.8402, lr_0 = 7.6436e-04
Validation auc = 0.707981
Epoch 165
Validation auc = 0.712207
Epoch 166
Validation auc = 0.713615
Epoch 167
Validation auc = 0.706573
Epoch 168
Validation auc = 0.710329
Epoch 169
Loss = 4.2041e-01, PNorm = 42.9160, GNorm = 2.5037, lr_0 = 7.5782e-04
Validation auc = 0.716432
Epoch 170
Validation auc = 0.716901
Epoch 171
Validation auc = 0.723944
Epoch 172
Validation auc = 0.735681
Epoch 173
Validation auc = 0.745540
Epoch 174
Loss = 3.7577e-01, PNorm = 43.0363, GNorm = 3.1675, lr_0 = 7.5134e-04
Validation auc = 0.748357
Epoch 175
Validation auc = 0.750704
Epoch 176
Validation auc = 0.739906
Epoch 177
Validation auc = 0.728638
Epoch 178
Validation auc = 0.730516
Epoch 179
Loss = 3.2812e-01, PNorm = 43.1468, GNorm = 2.8677, lr_0 = 7.4492e-04
Validation auc = 0.727700
Epoch 180
Validation auc = 0.745540
Epoch 181
Validation auc = 0.744601
Epoch 182
Validation auc = 0.746948
Epoch 183
Validation auc = 0.746479
Epoch 184
Loss = 3.6043e-01, PNorm = 43.2585, GNorm = 3.0121, lr_0 = 7.3855e-04
Validation auc = 0.738028
Epoch 185
Validation auc = 0.729108
Epoch 186
Validation auc = 0.725822
Epoch 187
Validation auc = 0.722066
Epoch 188
Validation auc = 0.714554
Epoch 189
Loss = 3.7800e-01, PNorm = 43.3535, GNorm = 2.7378, lr_0 = 7.3224e-04
Validation auc = 0.718779
Epoch 190
Validation auc = 0.722066
Epoch 191
Validation auc = 0.724883
Epoch 192
Validation auc = 0.722535
Epoch 193
Validation auc = 0.720657
Epoch 194
Loss = 3.5818e-01, PNorm = 43.4567, GNorm = 3.2224, lr_0 = 7.2598e-04
Validation auc = 0.717840
Epoch 195
Validation auc = 0.717371
Epoch 196
Validation auc = 0.718779
Epoch 197
Validation auc = 0.715962
Epoch 198
Validation auc = 0.717840
Epoch 199
Loss = 3.4056e-01, PNorm = 43.5553, GNorm = 2.2916, lr_0 = 7.1977e-04
Validation auc = 0.725822
Epoch 200
Validation auc = 0.733333
Epoch 201
Validation auc = 0.723944
Epoch 202
Validation auc = 0.723005
Epoch 203
Validation auc = 0.722535
Epoch 204
Loss = 3.3288e-01, PNorm = 43.6507, GNorm = 2.0318, lr_0 = 7.1362e-04
Validation auc = 0.722066
Epoch 205
Validation auc = 0.722535
Epoch 206
Validation auc = 0.725822
Epoch 207
Validation auc = 0.730986
Epoch 208
Validation auc = 0.740845
Epoch 209
Loss = 3.3900e-01, PNorm = 43.7688, GNorm = 2.3367, lr_0 = 7.0752e-04
Validation auc = 0.732864
Epoch 210
Validation auc = 0.723944
Epoch 211
Validation auc = 0.726291
Epoch 212
Validation auc = 0.730516
Epoch 213
Validation auc = 0.728169
Epoch 214
Loss = 3.6302e-01, PNorm = 43.8446, GNorm = 2.9859, lr_0 = 7.0147e-04
Validation auc = 0.748357
Epoch 215
Validation auc = 0.751643
Epoch 216
Validation auc = 0.755869
Epoch 217
Validation auc = 0.755399
Epoch 218
Validation auc = 0.749765
Epoch 219
Loss = 3.5437e-01, PNorm = 43.9305, GNorm = 2.6024, lr_0 = 6.9548e-04
Validation auc = 0.749765
Epoch 220
Validation auc = 0.745540
Epoch 221
Validation auc = 0.746948
Epoch 222
Validation auc = 0.747887
Epoch 223
Validation auc = 0.747418
Epoch 224
Loss = 2.8031e-01, PNorm = 44.0147, GNorm = 2.4544, lr_0 = 6.8953e-04
Validation auc = 0.749765
Epoch 225
Validation auc = 0.750235
Epoch 226
Validation auc = 0.757277
Epoch 227
Validation auc = 0.760094
Epoch 228
Validation auc = 0.768545
Epoch 229
Loss = 3.6740e-01, PNorm = 44.1040, GNorm = 2.4796, lr_0 = 6.8364e-04
Validation auc = 0.769484
Epoch 230
Validation auc = 0.770892
Epoch 231
Validation auc = 0.773709
Epoch 232
Validation auc = 0.765258
Epoch 233
Validation auc = 0.761972
Epoch 234
Loss = 2.5582e-01, PNorm = 44.1900, GNorm = 2.6549, lr_0 = 6.7779e-04
Validation auc = 0.756338
Epoch 235
Validation auc = 0.754930
Epoch 236
Validation auc = 0.752582
Epoch 237
Validation auc = 0.761033
Epoch 238
Validation auc = 0.761033
Epoch 239
Loss = 2.8032e-01, PNorm = 44.2854, GNorm = 2.3862, lr_0 = 6.7200e-04
Validation auc = 0.762441
Epoch 240
Validation auc = 0.761972
Epoch 241
Validation auc = 0.765258
Epoch 242
Validation auc = 0.756338
Epoch 243
Validation auc = 0.750704
Epoch 244
Loss = 2.8897e-01, PNorm = 44.3639, GNorm = 3.0296, lr_0 = 6.6625e-04
Validation auc = 0.737089
Epoch 245
Validation auc = 0.742723
Epoch 246
Validation auc = 0.746948
Epoch 247
Validation auc = 0.752113
Epoch 248
Validation auc = 0.749765
Epoch 249
Loss = 3.1562e-01, PNorm = 44.4432, GNorm = 1.9728, lr_0 = 6.6056e-04
Validation auc = 0.749765
Epoch 250
Validation auc = 0.753052
Epoch 251
Validation auc = 0.753052
Epoch 252
Validation auc = 0.746009
Epoch 253
Validation auc = 0.741784
Epoch 254
Loss = 3.7903e-01, PNorm = 44.5402, GNorm = 2.7568, lr_0 = 6.5491e-04
Validation auc = 0.737559
Epoch 255
Validation auc = 0.733333
Epoch 256
Validation auc = 0.734272
Epoch 257
Validation auc = 0.739906
Epoch 258
Validation auc = 0.749296
Epoch 259
Loss = 3.3796e-01, PNorm = 44.6455, GNorm = 1.8949, lr_0 = 6.4931e-04
Validation auc = 0.758685
Epoch 260
Validation auc = 0.754460
Epoch 261
Validation auc = 0.754460
Epoch 262
Validation auc = 0.750235
Epoch 263
Validation auc = 0.735211
Epoch 264
Loss = 2.6654e-01, PNorm = 44.7238, GNorm = 3.9596, lr_0 = 6.4376e-04
Validation auc = 0.725352
Epoch 265
Validation auc = 0.720188
Epoch 266
Validation auc = 0.719249
Epoch 267
Validation auc = 0.720657
Epoch 268
Validation auc = 0.725822
Epoch 269
Loss = 2.6060e-01, PNorm = 44.7848, GNorm = 1.9669, lr_0 = 6.3826e-04
Validation auc = 0.727230
Epoch 270
Validation auc = 0.728638
Epoch 271
Validation auc = 0.727700
Epoch 272
Validation auc = 0.730986
Epoch 273
Validation auc = 0.736620
Epoch 274
Loss = 2.0992e-01, PNorm = 44.8446, GNorm = 1.6835, lr_0 = 6.3280e-04
Validation auc = 0.746479
Epoch 275
Validation auc = 0.751643
Epoch 276
Validation auc = 0.752113
Epoch 277
Validation auc = 0.751174
Epoch 278
Validation auc = 0.753052
Epoch 279
Loss = 3.3659e-01, PNorm = 44.9187, GNorm = 1.9939, lr_0 = 6.2739e-04
Validation auc = 0.747418
Epoch 280
Validation auc = 0.742723
Epoch 281
Validation auc = 0.739437
Epoch 282
Validation auc = 0.737089
Epoch 283
Validation auc = 0.732394
Epoch 284
Loss = 3.1821e-01, PNorm = 44.9958, GNorm = 3.0341, lr_0 = 6.2203e-04
Validation auc = 0.727700
Epoch 285
Validation auc = 0.733333
Epoch 286
Validation auc = 0.726761
Epoch 287
Validation auc = 0.729108
Epoch 288
Validation auc = 0.731925
Epoch 289
Loss = 3.7322e-01, PNorm = 45.0684, GNorm = 2.0801, lr_0 = 6.1671e-04
Validation auc = 0.734742
Epoch 290
Validation auc = 0.736620
Epoch 291
Validation auc = 0.733333
Epoch 292
Validation auc = 0.731455
Epoch 293
Validation auc = 0.733803
Epoch 294
Loss = 2.2038e-01, PNorm = 45.1355, GNorm = 2.4326, lr_0 = 6.1144e-04
Validation auc = 0.736620
Epoch 295
Validation auc = 0.735681
Epoch 296
Validation auc = 0.732394
Epoch 297
Validation auc = 0.731925
Epoch 298
Validation auc = 0.733333
Epoch 299
Loss = 1.8042e-01, PNorm = 45.2102, GNorm = 2.2650, lr_0 = 6.0621e-04
Validation auc = 0.730047
Model 0 best validation auc = 0.773709 on epoch 231
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.927152
Ensemble test auc = 0.927152
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.564319
Epoch 1
Validation auc = 0.687793
Epoch 2
Validation auc = 0.679343
Epoch 3
Validation auc = 0.688263
Epoch 4
Loss = 1.4051e+00, PNorm = 40.6672, GNorm = 11.7362, lr_0 = 6.5000e-04
Validation auc = 0.707042
Epoch 5
Validation auc = 0.727230
Epoch 6
Validation auc = 0.702817
Epoch 7
Validation auc = 0.699531
Epoch 8
Validation auc = 0.690610
Epoch 9
Loss = 9.0930e-01, PNorm = 40.7167, GNorm = 8.7234, lr_0 = 9.9743e-04
Validation auc = 0.692958
Epoch 10
Validation auc = 0.653991
Epoch 11
Validation auc = 0.620188
Epoch 12
Validation auc = 0.623005
Epoch 13
Validation auc = 0.619718
Epoch 14
Loss = 8.4163e-01, PNorm = 40.7653, GNorm = 8.5683, lr_0 = 9.8890e-04
Validation auc = 0.683568
Epoch 15
Validation auc = 0.682160
Epoch 16
Validation auc = 0.720657
Epoch 17
Validation auc = 0.682629
Epoch 18
Validation auc = 0.700939
Epoch 19
Loss = 6.5092e-01, PNorm = 40.8110, GNorm = 4.1391, lr_0 = 9.8045e-04
Validation auc = 0.750235
Epoch 20
Validation auc = 0.746009
Epoch 21
Validation auc = 0.715493
Epoch 22
Validation auc = 0.694366
Epoch 23
Validation auc = 0.694836
Epoch 24
Loss = 7.0423e-01, PNorm = 40.8549, GNorm = 5.6547, lr_0 = 9.7207e-04
Validation auc = 0.710329
Epoch 25
Validation auc = 0.689671
Epoch 26
Validation auc = 0.665728
Epoch 27
Validation auc = 0.649765
Epoch 28
Validation auc = 0.646948
Epoch 29
Loss = 6.3085e-01, PNorm = 40.8841, GNorm = 4.9898, lr_0 = 9.6376e-04
Validation auc = 0.654460
Epoch 30
Validation auc = 0.662441
Epoch 31
Validation auc = 0.669953
Epoch 32
Validation auc = 0.658216
Epoch 33
Validation auc = 0.670423
Epoch 34
Loss = 6.9486e-01, PNorm = 40.9205, GNorm = 5.9975, lr_0 = 9.5552e-04
Validation auc = 0.672770
Epoch 35
Validation auc = 0.678404
Epoch 36
Validation auc = 0.697653
Epoch 37
Validation auc = 0.718779
Epoch 38
Validation auc = 0.688732
Epoch 39
Loss = 6.3885e-01, PNorm = 40.9647, GNorm = 4.9264, lr_0 = 9.4735e-04
Validation auc = 0.689202
Epoch 40
Validation auc = 0.726761
Epoch 41
Validation auc = 0.745070
Epoch 42
Validation auc = 0.736150
Epoch 43
Validation auc = 0.708920
Epoch 44
Loss = 4.8536e-01, PNorm = 41.0070, GNorm = 3.8567, lr_0 = 9.3925e-04
Validation auc = 0.700469
Epoch 45
Validation auc = 0.673709
Epoch 46
Validation auc = 0.660094
Epoch 47
Validation auc = 0.650704
Epoch 48
Validation auc = 0.646009
Epoch 49
Loss = 5.8854e-01, PNorm = 41.0510, GNorm = 5.6494, lr_0 = 9.3122e-04
Validation auc = 0.644131
Epoch 50
Validation auc = 0.646948
Epoch 51
Validation auc = 0.649765
Epoch 52
Validation auc = 0.635681
Epoch 53
Validation auc = 0.637559
Epoch 54
Loss = 6.0475e-01, PNorm = 41.1008, GNorm = 3.8984, lr_0 = 9.2326e-04
Validation auc = 0.648826
Epoch 55
Validation auc = 0.648826
Epoch 56
Validation auc = 0.646948
Epoch 57
Validation auc = 0.645540
Epoch 58
Validation auc = 0.647418
Epoch 59
Loss = 6.2247e-01, PNorm = 41.1582, GNorm = 4.4028, lr_0 = 9.1537e-04
Validation auc = 0.665258
Epoch 60
Validation auc = 0.674648
Epoch 61
Validation auc = 0.669953
Epoch 62
Validation auc = 0.664319
Epoch 63
Validation auc = 0.662441
Epoch 64
Loss = 5.2638e-01, PNorm = 41.1913, GNorm = 3.4416, lr_0 = 9.0754e-04
Validation auc = 0.669014
Epoch 65
Validation auc = 0.681221
Epoch 66
Validation auc = 0.665728
Epoch 67
Validation auc = 0.667606
Epoch 68
Validation auc = 0.669953
Epoch 69
Loss = 5.2159e-01, PNorm = 41.2450, GNorm = 3.2208, lr_0 = 8.9978e-04
Validation auc = 0.670892
Epoch 70
Validation auc = 0.668545
Epoch 71
Validation auc = 0.693897
Epoch 72
Validation auc = 0.674178
Epoch 73
Validation auc = 0.659155
Epoch 74
Loss = 5.5974e-01, PNorm = 41.2845, GNorm = 2.7567, lr_0 = 8.9209e-04
Validation auc = 0.647418
Epoch 75
Validation auc = 0.632394
Epoch 76
Validation auc = 0.635681
Epoch 77
Validation auc = 0.644131
Epoch 78
Validation auc = 0.653991
Epoch 79
Loss = 5.5882e-01, PNorm = 41.3513, GNorm = 3.4976, lr_0 = 8.8447e-04
Validation auc = 0.656808
Epoch 80
Validation auc = 0.654460
Epoch 81
Validation auc = 0.657277
Epoch 82
Validation auc = 0.656338
Epoch 83
Validation auc = 0.656808
Epoch 84
Loss = 4.9009e-01, PNorm = 41.4135, GNorm = 3.0505, lr_0 = 8.7691e-04
Validation auc = 0.653521
Epoch 85
Validation auc = 0.651174
Epoch 86
Validation auc = 0.656808
Epoch 87
Validation auc = 0.669953
Epoch 88
Validation auc = 0.681221
Epoch 89
Loss = 5.8163e-01, PNorm = 41.4696, GNorm = 2.6076, lr_0 = 8.6941e-04
Validation auc = 0.671362
Epoch 90
Validation auc = 0.671362
Epoch 91
Validation auc = 0.673709
Epoch 92
Validation auc = 0.667606
Epoch 93
Validation auc = 0.662441
Epoch 94
Loss = 6.2416e-01, PNorm = 41.5626, GNorm = 3.7212, lr_0 = 8.6198e-04
Validation auc = 0.657746
Epoch 95
Validation auc = 0.655399
Epoch 96
Validation auc = 0.650235
Epoch 97
Validation auc = 0.646479
Epoch 98
Validation auc = 0.650704
Epoch 99
Loss = 5.6097e-01, PNorm = 41.6217, GNorm = 3.2718, lr_0 = 8.5461e-04
Validation auc = 0.662441
Epoch 100
Validation auc = 0.678404
Epoch 101
Validation auc = 0.686854
Epoch 102
Validation auc = 0.705164
Epoch 103
Validation auc = 0.707512
Epoch 104
Loss = 4.3506e-01, PNorm = 41.7065, GNorm = 2.9852, lr_0 = 8.4730e-04
Validation auc = 0.709859
Epoch 105
Validation auc = 0.709859
Epoch 106
Validation auc = 0.712676
Epoch 107
Validation auc = 0.710798
Epoch 108
Validation auc = 0.722066
Epoch 109
Loss = 4.6150e-01, PNorm = 41.7788, GNorm = 2.9168, lr_0 = 8.4006e-04
Validation auc = 0.724413
Epoch 110
Validation auc = 0.724883
Epoch 111
Validation auc = 0.725822
Epoch 112
Validation auc = 0.717840
Epoch 113
Validation auc = 0.714085
Epoch 114
Loss = 5.3994e-01, PNorm = 41.8744, GNorm = 3.8980, lr_0 = 8.3288e-04
Validation auc = 0.714085
Epoch 115
Validation auc = 0.705634
Epoch 116
Validation auc = 0.713615
Epoch 117
Validation auc = 0.736620
Epoch 118
Validation auc = 0.756808
Epoch 119
Loss = 5.4242e-01, PNorm = 41.9537, GNorm = 2.9311, lr_0 = 8.2576e-04
Validation auc = 0.768545
Epoch 120
Validation auc = 0.764789
Epoch 121
Validation auc = 0.753521
Epoch 122
Validation auc = 0.745070
Epoch 123
Validation auc = 0.739437
Epoch 124
Loss = 4.1503e-01, PNorm = 42.0662, GNorm = 2.6455, lr_0 = 8.1870e-04
Validation auc = 0.732864
Epoch 125
Validation auc = 0.730516
Epoch 126
Validation auc = 0.731455
Epoch 127
Validation auc = 0.727700
Epoch 128
Validation auc = 0.730986
Epoch 129
Loss = 3.8873e-01, PNorm = 42.1833, GNorm = 2.6015, lr_0 = 8.1170e-04
Validation auc = 0.727700
Epoch 130
Validation auc = 0.708920
Epoch 131
Validation auc = 0.706103
Epoch 132
Validation auc = 0.703756
Epoch 133
Validation auc = 0.705634
Epoch 134
Loss = 3.4970e-01, PNorm = 42.2865, GNorm = 2.1979, lr_0 = 8.0476e-04
Validation auc = 0.707512
Epoch 135
Validation auc = 0.707512
Epoch 136
Validation auc = 0.699061
Epoch 137
Validation auc = 0.696244
Epoch 138
Validation auc = 0.694836
Epoch 139
Loss = 3.5926e-01, PNorm = 42.3642, GNorm = 2.6086, lr_0 = 7.9788e-04
Validation auc = 0.695775
Epoch 140
Validation auc = 0.700469
Epoch 141
Validation auc = 0.702817
Epoch 142
Validation auc = 0.707981
Epoch 143
Validation auc = 0.708920
Epoch 144
Loss = 3.5823e-01, PNorm = 42.4626, GNorm = 2.9428, lr_0 = 7.9106e-04
Validation auc = 0.715023
Epoch 145
Validation auc = 0.728169
Epoch 146
Validation auc = 0.735681
Epoch 147
Validation auc = 0.730047
Epoch 148
Validation auc = 0.735681
Epoch 149
Loss = 4.0792e-01, PNorm = 42.5467, GNorm = 3.5513, lr_0 = 7.8430e-04
Validation auc = 0.728169
Epoch 150
Validation auc = 0.714085
Epoch 151
Validation auc = 0.711268
Epoch 152
Validation auc = 0.700469
Epoch 153
Validation auc = 0.688263
Epoch 154
Loss = 4.4545e-01, PNorm = 42.6251, GNorm = 3.1155, lr_0 = 7.7759e-04
Validation auc = 0.681221
Epoch 155
Validation auc = 0.681221
Epoch 156
Validation auc = 0.676526
Epoch 157
Validation auc = 0.685446
Epoch 158
Validation auc = 0.697653
Epoch 159
Loss = 3.7058e-01, PNorm = 42.6939, GNorm = 2.3437, lr_0 = 7.7095e-04
Validation auc = 0.704695
Epoch 160
Validation auc = 0.710798
Epoch 161
Validation auc = 0.711268
Epoch 162
Validation auc = 0.709859
Epoch 163
Validation auc = 0.706573
Epoch 164
Loss = 3.3321e-01, PNorm = 42.7721, GNorm = 3.6732, lr_0 = 7.6436e-04
Validation auc = 0.709390
Epoch 165
Validation auc = 0.709390
Epoch 166
Validation auc = 0.707512
Epoch 167
Validation auc = 0.710329
Epoch 168
Validation auc = 0.718779
Epoch 169
Loss = 3.2255e-01, PNorm = 42.8506, GNorm = 2.5339, lr_0 = 7.5782e-04
Validation auc = 0.730516
Epoch 170
Validation auc = 0.737559
Epoch 171
Validation auc = 0.741315
Epoch 172
Validation auc = 0.743662
Epoch 173
Validation auc = 0.738498
Epoch 174
Loss = 3.3780e-01, PNorm = 42.9225, GNorm = 2.9563, lr_0 = 7.5134e-04
Validation auc = 0.731455
Epoch 175
Validation auc = 0.728638
Epoch 176
Validation auc = 0.728638
Epoch 177
Validation auc = 0.725352
Epoch 178
Validation auc = 0.732864
Epoch 179
Loss = 3.8685e-01, PNorm = 43.0155, GNorm = 2.7695, lr_0 = 7.4492e-04
Validation auc = 0.732394
Epoch 180
Validation auc = 0.735681
Epoch 181
Validation auc = 0.738498
Epoch 182
Validation auc = 0.745070
Epoch 183
Validation auc = 0.743662
Epoch 184
Loss = 3.6530e-01, PNorm = 43.1054, GNorm = 2.9411, lr_0 = 7.3855e-04
Validation auc = 0.738498
Epoch 185
Validation auc = 0.733803
Epoch 186
Validation auc = 0.729577
Epoch 187
Validation auc = 0.732394
Epoch 188
Validation auc = 0.731925
Epoch 189
Loss = 3.4936e-01, PNorm = 43.1918, GNorm = 2.5728, lr_0 = 7.3224e-04
Validation auc = 0.730986
Epoch 190
Validation auc = 0.731925
Epoch 191
Validation auc = 0.731925
Epoch 192
Validation auc = 0.731925
Epoch 193
Validation auc = 0.733803
Epoch 194
Loss = 3.4340e-01, PNorm = 43.2851, GNorm = 2.7547, lr_0 = 7.2598e-04
Validation auc = 0.735211
Epoch 195
Validation auc = 0.745540
Epoch 196
Validation auc = 0.744601
Epoch 197
Validation auc = 0.747887
Epoch 198
Validation auc = 0.741315
Epoch 199
Loss = 3.3705e-01, PNorm = 43.3819, GNorm = 2.7729, lr_0 = 7.1977e-04
Validation auc = 0.746009
Epoch 200
Validation auc = 0.744601
Epoch 201
Validation auc = 0.738028
Epoch 202
Validation auc = 0.735681
Epoch 203
Validation auc = 0.731455
Epoch 204
Loss = 2.8032e-01, PNorm = 43.4714, GNorm = 2.1922, lr_0 = 7.1362e-04
Validation auc = 0.727700
Epoch 205
Validation auc = 0.724883
Epoch 206
Validation auc = 0.727230
Epoch 207
Validation auc = 0.727230
Epoch 208
Validation auc = 0.725822
Epoch 209
Loss = 3.2756e-01, PNorm = 43.5704, GNorm = 1.8834, lr_0 = 7.0752e-04
Validation auc = 0.726761
Epoch 210
Validation auc = 0.722066
Epoch 211
Validation auc = 0.723474
Epoch 212
Validation auc = 0.717371
Epoch 213
Validation auc = 0.718779
Epoch 214
Loss = 3.5166e-01, PNorm = 43.6739, GNorm = 3.3102, lr_0 = 7.0147e-04
Validation auc = 0.723474
Epoch 215
Validation auc = 0.727230
Epoch 216
Validation auc = 0.728638
Epoch 217
Validation auc = 0.728638
Epoch 218
Validation auc = 0.741784
Epoch 219
Loss = 2.2628e-01, PNorm = 43.7585, GNorm = 1.9857, lr_0 = 6.9548e-04
Validation auc = 0.753991
Epoch 220
Validation auc = 0.755399
Epoch 221
Validation auc = 0.755869
Epoch 222
Validation auc = 0.754930
Epoch 223
Validation auc = 0.750704
Epoch 224
Loss = 2.4273e-01, PNorm = 43.8471, GNorm = 1.8774, lr_0 = 6.8953e-04
Validation auc = 0.747887
Epoch 225
Validation auc = 0.753052
Epoch 226
Validation auc = 0.756338
Epoch 227
Validation auc = 0.753521
Epoch 228
Validation auc = 0.748357
Epoch 229
Loss = 2.7182e-01, PNorm = 43.9272, GNorm = 2.7130, lr_0 = 6.8364e-04
Validation auc = 0.742723
Epoch 230
Validation auc = 0.737559
Epoch 231
Validation auc = 0.736150
Epoch 232
Validation auc = 0.734742
Epoch 233
Validation auc = 0.732864
Epoch 234
Loss = 2.5148e-01, PNorm = 44.0142, GNorm = 1.9918, lr_0 = 6.7779e-04
Validation auc = 0.729577
Epoch 235
Validation auc = 0.730516
Epoch 236
Validation auc = 0.734742
Epoch 237
Validation auc = 0.736150
Epoch 238
Validation auc = 0.739437
Epoch 239
Loss = 2.0977e-01, PNorm = 44.1101, GNorm = 2.8347, lr_0 = 6.7200e-04
Validation auc = 0.733803
Epoch 240
Validation auc = 0.733333
Epoch 241
Validation auc = 0.732394
Epoch 242
Validation auc = 0.723474
Epoch 243
Validation auc = 0.730986
Epoch 244
Loss = 3.7964e-01, PNorm = 44.1979, GNorm = 2.6817, lr_0 = 6.6625e-04
Validation auc = 0.735211
Epoch 245
Validation auc = 0.732394
Epoch 246
Validation auc = 0.730986
Epoch 247
Validation auc = 0.734742
Epoch 248
Validation auc = 0.731925
Epoch 249
Loss = 3.3532e-01, PNorm = 44.2639, GNorm = 2.1740, lr_0 = 6.6056e-04
Validation auc = 0.736620
Epoch 250
Validation auc = 0.739437
Epoch 251
Validation auc = 0.738498
Epoch 252
Validation auc = 0.742254
Epoch 253
Validation auc = 0.746009
Epoch 254
Loss = 2.6617e-01, PNorm = 44.3378, GNorm = 2.7253, lr_0 = 6.5491e-04
Validation auc = 0.748357
Epoch 255
Validation auc = 0.750235
Epoch 256
Validation auc = 0.753052
Epoch 257
Validation auc = 0.754460
Epoch 258
Validation auc = 0.750704
Epoch 259
Loss = 4.4055e-01, PNorm = 44.4208, GNorm = 4.6089, lr_0 = 6.4931e-04
Validation auc = 0.747887
Epoch 260
Validation auc = 0.743192
Epoch 261
Validation auc = 0.737089
Epoch 262
Validation auc = 0.729108
Epoch 263
Validation auc = 0.723005
Epoch 264
Loss = 3.5164e-01, PNorm = 44.4848, GNorm = 2.3530, lr_0 = 6.4376e-04
Validation auc = 0.723474
Epoch 265
Validation auc = 0.723005
Epoch 266
Validation auc = 0.728169
Epoch 267
Validation auc = 0.731925
Epoch 268
Validation auc = 0.739906
Epoch 269
Loss = 2.2422e-01, PNorm = 44.5657, GNorm = 1.8320, lr_0 = 6.3826e-04
Validation auc = 0.746479
Epoch 270
Validation auc = 0.750704
Epoch 271
Validation auc = 0.747887
Epoch 272
Validation auc = 0.748826
Epoch 273
Validation auc = 0.750704
Epoch 274
Loss = 2.1286e-01, PNorm = 44.6566, GNorm = 1.4141, lr_0 = 6.3280e-04
Validation auc = 0.754460
Epoch 275
Validation auc = 0.753521
Epoch 276
Validation auc = 0.745540
Epoch 277
Validation auc = 0.740376
Epoch 278
Validation auc = 0.741784
Epoch 279
Loss = 2.7187e-01, PNorm = 44.7222, GNorm = 1.8185, lr_0 = 6.2739e-04
Validation auc = 0.747887
Epoch 280
Validation auc = 0.753052
Epoch 281
Validation auc = 0.752113
Epoch 282
Validation auc = 0.753991
Epoch 283
Validation auc = 0.754930
Epoch 284
Loss = 2.7825e-01, PNorm = 44.7941, GNorm = 4.0537, lr_0 = 6.2203e-04
Validation auc = 0.758216
Epoch 285
Validation auc = 0.757746
Epoch 286
Validation auc = 0.757277
Epoch 287
Validation auc = 0.746479
Epoch 288
Validation auc = 0.749765
Epoch 289
Loss = 2.1320e-01, PNorm = 44.8638, GNorm = 2.3243, lr_0 = 6.1671e-04
Validation auc = 0.747887
Epoch 290
Validation auc = 0.748826
Epoch 291
Validation auc = 0.754930
Epoch 292
Validation auc = 0.758685
Epoch 293
Validation auc = 0.750235
Epoch 294
Loss = 2.4331e-01, PNorm = 44.9449, GNorm = 2.9290, lr_0 = 6.1144e-04
Validation auc = 0.746009
Epoch 295
Validation auc = 0.744601
Epoch 296
Validation auc = 0.746479
Epoch 297
Validation auc = 0.742723
Epoch 298
Validation auc = 0.739437
Epoch 299
Loss = 2.3498e-01, PNorm = 45.0247, GNorm = 2.3303, lr_0 = 6.0621e-04
Validation auc = 0.735211
Model 0 best validation auc = 0.768545 on epoch 119
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.913305
Ensemble test auc = 0.913305
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.681221
Epoch 1
Validation auc = 0.677465
Epoch 2
Validation auc = 0.669953
Epoch 3
Validation auc = 0.662911
Epoch 4
Loss = 1.1727e+00, PNorm = 40.6676, GNorm = 14.6704, lr_0 = 6.5000e-04
Validation auc = 0.638028
Epoch 5
Validation auc = 0.654930
Epoch 6
Validation auc = 0.653991
Epoch 7
Validation auc = 0.648357
Epoch 8
Validation auc = 0.649296
Epoch 9
Loss = 8.6141e-01, PNorm = 40.7109, GNorm = 9.1591, lr_0 = 9.9743e-04
Validation auc = 0.627230
Epoch 10
Validation auc = 0.619718
Epoch 11
Validation auc = 0.644601
Epoch 12
Validation auc = 0.622535
Epoch 13
Validation auc = 0.618310
Epoch 14
Loss = 7.2814e-01, PNorm = 40.7650, GNorm = 6.6048, lr_0 = 9.8890e-04
Validation auc = 0.599061
Epoch 15
Validation auc = 0.637089
Epoch 16
Validation auc = 0.651174
Epoch 17
Validation auc = 0.652582
Epoch 18
Validation auc = 0.724413
Epoch 19
Loss = 6.7291e-01, PNorm = 40.8132, GNorm = 6.3667, lr_0 = 9.8045e-04
Validation auc = 0.657746
Epoch 20
Validation auc = 0.662441
Epoch 21
Validation auc = 0.662911
Epoch 22
Validation auc = 0.681690
Epoch 23
Validation auc = 0.687324
Epoch 24
Loss = 7.9424e-01, PNorm = 40.8495, GNorm = 7.9910, lr_0 = 9.7207e-04
Validation auc = 0.672300
Epoch 25
Validation auc = 0.674178
Epoch 26
Validation auc = 0.658685
Epoch 27
Validation auc = 0.655399
Epoch 28
Validation auc = 0.656338
Epoch 29
Loss = 5.4907e-01, PNorm = 40.8834, GNorm = 4.3092, lr_0 = 9.6376e-04
Validation auc = 0.656808
Epoch 30
Validation auc = 0.641784
Epoch 31
Validation auc = 0.609859
Epoch 32
Validation auc = 0.600469
Epoch 33
Validation auc = 0.607981
Epoch 34
Loss = 7.4850e-01, PNorm = 40.9016, GNorm = 6.1751, lr_0 = 9.5552e-04
Validation auc = 0.623474
Epoch 35
Validation auc = 0.663850
Epoch 36
Validation auc = 0.698592
Epoch 37
Validation auc = 0.701878
Epoch 38
Validation auc = 0.703756
Epoch 39
Loss = 6.8735e-01, PNorm = 40.9312, GNorm = 3.6573, lr_0 = 9.4735e-04
Validation auc = 0.728638
Epoch 40
Validation auc = 0.763850
Epoch 41
Validation auc = 0.732394
Epoch 42
Validation auc = 0.700469
Epoch 43
Validation auc = 0.695775
Epoch 44
Loss = 6.6924e-01, PNorm = 40.9551, GNorm = 5.3660, lr_0 = 9.3925e-04
Validation auc = 0.681221
Epoch 45
Validation auc = 0.681690
Epoch 46
Validation auc = 0.678404
Epoch 47
Validation auc = 0.654930
Epoch 48
Validation auc = 0.659624
Epoch 49
Loss = 6.4416e-01, PNorm = 40.9989, GNorm = 4.5010, lr_0 = 9.3122e-04
Validation auc = 0.665258
Epoch 50
Validation auc = 0.669484
Epoch 51
Validation auc = 0.659155
Epoch 52
Validation auc = 0.645070
Epoch 53
Validation auc = 0.623944
Epoch 54
Loss = 5.9989e-01, PNorm = 41.0301, GNorm = 5.4409, lr_0 = 9.2326e-04
Validation auc = 0.617371
Epoch 55
Validation auc = 0.619718
Epoch 56
Validation auc = 0.618310
Epoch 57
Validation auc = 0.618310
Epoch 58
Validation auc = 0.631455
Epoch 59
Loss = 5.9493e-01, PNorm = 41.0679, GNorm = 4.3275, lr_0 = 9.1537e-04
Validation auc = 0.637089
Epoch 60
Validation auc = 0.631925
Epoch 61
Validation auc = 0.631455
Epoch 62
Validation auc = 0.641315
Epoch 63
Validation auc = 0.656338
Epoch 64
Loss = 6.0904e-01, PNorm = 41.1015, GNorm = 3.8266, lr_0 = 9.0754e-04
Validation auc = 0.650235
Epoch 65
Validation auc = 0.646009
Epoch 66
Validation auc = 0.636620
Epoch 67
Validation auc = 0.631925
Epoch 68
Validation auc = 0.638028
Epoch 69
Loss = 4.7100e-01, PNorm = 41.1589, GNorm = 3.7963, lr_0 = 8.9978e-04
Validation auc = 0.646479
Epoch 70
Validation auc = 0.649765
Epoch 71
Validation auc = 0.637559
Epoch 72
Validation auc = 0.633333
Epoch 73
Validation auc = 0.631455
Epoch 74
Loss = 5.6367e-01, PNorm = 41.2120, GNorm = 4.3209, lr_0 = 8.9209e-04
Validation auc = 0.630516
Epoch 75
Validation auc = 0.634742
Epoch 76
Validation auc = 0.641315
Epoch 77
Validation auc = 0.637089
Epoch 78
Validation auc = 0.634742
Epoch 79
Loss = 5.2709e-01, PNorm = 41.2384, GNorm = 3.4510, lr_0 = 8.8447e-04
Validation auc = 0.626761
Epoch 80
Validation auc = 0.624883
Epoch 81
Validation auc = 0.625822
Epoch 82
Validation auc = 0.630516
Epoch 83
Validation auc = 0.636620
Epoch 84
Loss = 4.4689e-01, PNorm = 41.3026, GNorm = 2.6538, lr_0 = 8.7691e-04
Validation auc = 0.635681
Epoch 85
Validation auc = 0.646479
Epoch 86
Validation auc = 0.648826
Epoch 87
Validation auc = 0.647418
Epoch 88
Validation auc = 0.653052
Epoch 89
Loss = 5.0990e-01, PNorm = 41.3538, GNorm = 3.5241, lr_0 = 8.6941e-04
Validation auc = 0.654930
Epoch 90
Validation auc = 0.658685
Epoch 91
Validation auc = 0.666197
Epoch 92
Validation auc = 0.665258
Epoch 93
Validation auc = 0.669953
Epoch 94
Loss = 4.5925e-01, PNorm = 41.4165, GNorm = 2.3186, lr_0 = 8.6198e-04
Validation auc = 0.666667
Epoch 95
Validation auc = 0.661033
Epoch 96
Validation auc = 0.647887
Epoch 97
Validation auc = 0.637559
Epoch 98
Validation auc = 0.630047
Epoch 99
Loss = 5.3186e-01, PNorm = 41.4778, GNorm = 3.0300, lr_0 = 8.5461e-04
Validation auc = 0.626761
Epoch 100
Validation auc = 0.625822
Epoch 101
Validation auc = 0.619718
Epoch 102
Validation auc = 0.628169
Epoch 103
Validation auc = 0.625822
Epoch 104
Loss = 5.0644e-01, PNorm = 41.5647, GNorm = 2.6269, lr_0 = 8.4730e-04
Validation auc = 0.626291
Epoch 105
Validation auc = 0.631925
Epoch 106
Validation auc = 0.637559
Epoch 107
Validation auc = 0.650235
Epoch 108
Validation auc = 0.660094
Epoch 109
Loss = 4.2352e-01, PNorm = 41.6538, GNorm = 2.4346, lr_0 = 8.4006e-04
Validation auc = 0.651643
Epoch 110
Validation auc = 0.643662
Epoch 111
Validation auc = 0.650235
Epoch 112
Validation auc = 0.652113
Epoch 113
Validation auc = 0.646009
Epoch 114
Loss = 5.1174e-01, PNorm = 41.7315, GNorm = 2.9920, lr_0 = 8.3288e-04
Validation auc = 0.652113
Epoch 115
Validation auc = 0.652113
Epoch 116
Validation auc = 0.646479
Epoch 117
Validation auc = 0.649765
Epoch 118
Validation auc = 0.650704
Epoch 119
Loss = 4.2323e-01, PNorm = 41.8124, GNorm = 2.3519, lr_0 = 8.2576e-04
Validation auc = 0.649296
Epoch 120
Validation auc = 0.647418
Epoch 121
Validation auc = 0.648826
Epoch 122
Validation auc = 0.643662
Epoch 123
Validation auc = 0.640376
Epoch 124
Loss = 4.2543e-01, PNorm = 41.8969, GNorm = 2.5333, lr_0 = 8.1870e-04
Validation auc = 0.644131
Epoch 125
Validation auc = 0.660094
Epoch 126
Validation auc = 0.658216
Epoch 127
Validation auc = 0.659624
Epoch 128
Validation auc = 0.659624
Epoch 129
Loss = 4.1787e-01, PNorm = 41.9872, GNorm = 1.8893, lr_0 = 8.1170e-04
Validation auc = 0.668075
Epoch 130
Validation auc = 0.680282
Epoch 131
Validation auc = 0.689671
Epoch 132
Validation auc = 0.694836
Epoch 133
Validation auc = 0.691549
Epoch 134
Loss = 5.1003e-01, PNorm = 42.0893, GNorm = 4.1963, lr_0 = 8.0476e-04
Validation auc = 0.676056
Epoch 135
Validation auc = 0.681221
Epoch 136
Validation auc = 0.684507
Epoch 137
Validation auc = 0.676526
Epoch 138
Validation auc = 0.679812
Epoch 139
Loss = 4.4795e-01, PNorm = 42.1844, GNorm = 2.8275, lr_0 = 7.9788e-04
Validation auc = 0.677465
Epoch 140
Validation auc = 0.690610
Epoch 141
Validation auc = 0.702347
Epoch 142
Validation auc = 0.684977
Epoch 143
Validation auc = 0.684507
Epoch 144
Loss = 4.3959e-01, PNorm = 42.2816, GNorm = 2.5273, lr_0 = 7.9106e-04
Validation auc = 0.687324
Epoch 145
Validation auc = 0.692019
Epoch 146
Validation auc = 0.710798
Epoch 147
Validation auc = 0.718779
Epoch 148
Validation auc = 0.721596
Epoch 149
Loss = 3.9867e-01, PNorm = 42.3744, GNorm = 2.1225, lr_0 = 7.8430e-04
Validation auc = 0.719718
Epoch 150
Validation auc = 0.723944
Epoch 151
Validation auc = 0.734272
Epoch 152
Validation auc = 0.740376
Epoch 153
Validation auc = 0.748826
Epoch 154
Loss = 5.2541e-01, PNorm = 42.4580, GNorm = 3.9638, lr_0 = 7.7759e-04
Validation auc = 0.757277
Epoch 155
Validation auc = 0.744601
Epoch 156
Validation auc = 0.729577
Epoch 157
Validation auc = 0.718779
Epoch 158
Validation auc = 0.713146
Epoch 159
Loss = 4.0985e-01, PNorm = 42.5545, GNorm = 2.7254, lr_0 = 7.7095e-04
Validation auc = 0.711737
Epoch 160
Validation auc = 0.718310
Epoch 161
Validation auc = 0.725822
Epoch 162
Validation auc = 0.731925
Epoch 163
Validation auc = 0.733333
Epoch 164
Loss = 4.2791e-01, PNorm = 42.6407, GNorm = 2.3567, lr_0 = 7.6436e-04
Validation auc = 0.730047
Epoch 165
Validation auc = 0.719718
Epoch 166
Validation auc = 0.715962
Epoch 167
Validation auc = 0.708920
Epoch 168
Validation auc = 0.703756
Epoch 169
Loss = 4.0718e-01, PNorm = 42.7321, GNorm = 2.4523, lr_0 = 7.5782e-04
Validation auc = 0.705164
Epoch 170
Validation auc = 0.700939
Epoch 171
Validation auc = 0.708920
Epoch 172
Validation auc = 0.715493
Epoch 173
Validation auc = 0.732394
Epoch 174
Loss = 4.5649e-01, PNorm = 42.8361, GNorm = 3.3361, lr_0 = 7.5134e-04
Validation auc = 0.737089
Epoch 175
Validation auc = 0.735211
Epoch 176
Validation auc = 0.730047
Epoch 177
Validation auc = 0.723944
Epoch 178
Validation auc = 0.707512
Epoch 179
Loss = 4.6537e-01, PNorm = 42.9396, GNorm = 3.4402, lr_0 = 7.4492e-04
Validation auc = 0.700469
Epoch 180
Validation auc = 0.704695
Epoch 181
Validation auc = 0.706573
Epoch 182
Validation auc = 0.722535
Epoch 183
Validation auc = 0.742723
Epoch 184
Loss = 4.4325e-01, PNorm = 43.0437, GNorm = 3.9401, lr_0 = 7.3855e-04
Validation auc = 0.746948
Epoch 185
Validation auc = 0.736620
Epoch 186
Validation auc = 0.716901
Epoch 187
Validation auc = 0.703286
Epoch 188
Validation auc = 0.693897
Epoch 189
Loss = 3.7391e-01, PNorm = 43.1419, GNorm = 2.0179, lr_0 = 7.3224e-04
Validation auc = 0.689671
Epoch 190
Validation auc = 0.697653
Epoch 191
Validation auc = 0.697653
Epoch 192
Validation auc = 0.702817
Epoch 193
Validation auc = 0.711268
Epoch 194
Loss = 3.6545e-01, PNorm = 43.2292, GNorm = 3.6288, lr_0 = 7.2598e-04
Validation auc = 0.723944
Epoch 195
Validation auc = 0.732864
Epoch 196
Validation auc = 0.732864
Epoch 197
Validation auc = 0.725822
Epoch 198
Validation auc = 0.725822
Epoch 199
Loss = 3.3024e-01, PNorm = 43.3228, GNorm = 2.1905, lr_0 = 7.1977e-04
Validation auc = 0.723005
Epoch 200
Validation auc = 0.724883
Epoch 201
Validation auc = 0.726291
Epoch 202
Validation auc = 0.727700
Epoch 203
Validation auc = 0.730047
Epoch 204
Loss = 4.2872e-01, PNorm = 43.4180, GNorm = 1.7103, lr_0 = 7.1362e-04
Validation auc = 0.731925
Epoch 205
Validation auc = 0.738028
Epoch 206
Validation auc = 0.738028
Epoch 207
Validation auc = 0.731925
Epoch 208
Validation auc = 0.725352
Epoch 209
Loss = 3.1306e-01, PNorm = 43.5044, GNorm = 2.8695, lr_0 = 7.0752e-04
Validation auc = 0.717840
Epoch 210
Validation auc = 0.711737
Epoch 211
Validation auc = 0.708920
Epoch 212
Validation auc = 0.713615
Epoch 213
Validation auc = 0.710329
Epoch 214
Loss = 2.8018e-01, PNorm = 43.5930, GNorm = 2.2115, lr_0 = 7.0147e-04
Validation auc = 0.704695
Epoch 215
Validation auc = 0.707512
Epoch 216
Validation auc = 0.710798
Epoch 217
Validation auc = 0.714554
Epoch 218
Validation auc = 0.715962
Epoch 219
Loss = 2.9224e-01, PNorm = 43.6914, GNorm = 2.0517, lr_0 = 6.9548e-04
Validation auc = 0.723474
Epoch 220
Validation auc = 0.723005
Epoch 221
Validation auc = 0.723005
Epoch 222
Validation auc = 0.719249
Epoch 223
Validation auc = 0.720188
Epoch 224
Loss = 3.1945e-01, PNorm = 43.7872, GNorm = 2.2772, lr_0 = 6.8953e-04
Validation auc = 0.723944
Epoch 225
Validation auc = 0.719249
Epoch 226
Validation auc = 0.716432
Epoch 227
Validation auc = 0.713146
Epoch 228
Validation auc = 0.708451
Epoch 229
Loss = 3.7921e-01, PNorm = 43.8907, GNorm = 3.8619, lr_0 = 6.8364e-04
Validation auc = 0.713146
Epoch 230
Validation auc = 0.720657
Epoch 231
Validation auc = 0.733333
Epoch 232
Validation auc = 0.748357
Epoch 233
Validation auc = 0.745070
Epoch 234
Loss = 2.9535e-01, PNorm = 43.9904, GNorm = 2.2146, lr_0 = 6.7779e-04
Validation auc = 0.729577
Epoch 235
Validation auc = 0.710798
Epoch 236
Validation auc = 0.710329
Epoch 237
Validation auc = 0.714085
Epoch 238
Validation auc = 0.710329
Epoch 239
Loss = 3.3344e-01, PNorm = 44.0809, GNorm = 3.4018, lr_0 = 6.7200e-04
Validation auc = 0.714554
Epoch 240
Validation auc = 0.722535
Epoch 241
Validation auc = 0.728169
Epoch 242
Validation auc = 0.738028
Epoch 243
Validation auc = 0.733333
Epoch 244
Loss = 2.3145e-01, PNorm = 44.1781, GNorm = 2.0004, lr_0 = 6.6625e-04
Validation auc = 0.724883
Epoch 245
Validation auc = 0.713615
Epoch 246
Validation auc = 0.713615
Epoch 247
Validation auc = 0.711737
Epoch 248
Validation auc = 0.712676
Epoch 249
Loss = 2.8079e-01, PNorm = 44.2733, GNorm = 2.0464, lr_0 = 6.6056e-04
Validation auc = 0.710798
Epoch 250
Validation auc = 0.715962
Epoch 251
Validation auc = 0.723474
Epoch 252
Validation auc = 0.732394
Epoch 253
Validation auc = 0.731925
Epoch 254
Loss = 2.6655e-01, PNorm = 44.3600, GNorm = 3.6381, lr_0 = 6.5491e-04
Validation auc = 0.727230
Epoch 255
Validation auc = 0.723474
Epoch 256
Validation auc = 0.722066
Epoch 257
Validation auc = 0.723005
Epoch 258
Validation auc = 0.728169
Epoch 259
Loss = 2.9938e-01, PNorm = 44.4516, GNorm = 3.5554, lr_0 = 6.4931e-04
Validation auc = 0.730047
Epoch 260
Validation auc = 0.732394
Epoch 261
Validation auc = 0.735211
Epoch 262
Validation auc = 0.741784
Epoch 263
Validation auc = 0.739906
Epoch 264
Loss = 3.0951e-01, PNorm = 44.5336, GNorm = 4.1220, lr_0 = 6.4376e-04
Validation auc = 0.734742
Epoch 265
Validation auc = 0.727700
Epoch 266
Validation auc = 0.721127
Epoch 267
Validation auc = 0.720188
Epoch 268
Validation auc = 0.721127
Epoch 269
Loss = 3.5341e-01, PNorm = 44.6227, GNorm = 3.4183, lr_0 = 6.3826e-04
Validation auc = 0.720657
Epoch 270
Validation auc = 0.720188
Epoch 271
Validation auc = 0.719718
Epoch 272
Validation auc = 0.726291
Epoch 273
Validation auc = 0.724413
Epoch 274
Loss = 3.0888e-01, PNorm = 44.7005, GNorm = 2.2725, lr_0 = 6.3280e-04
Validation auc = 0.727230
Epoch 275
Validation auc = 0.729577
Epoch 276
Validation auc = 0.731455
Epoch 277
Validation auc = 0.720657
Epoch 278
Validation auc = 0.723474
Epoch 279
Loss = 2.8115e-01, PNorm = 44.7759, GNorm = 2.6436, lr_0 = 6.2739e-04
Validation auc = 0.727700
Epoch 280
Validation auc = 0.725352
Epoch 281
Validation auc = 0.723005
Epoch 282
Validation auc = 0.722535
Epoch 283
Validation auc = 0.729577
Epoch 284
Loss = 2.7123e-01, PNorm = 44.8542, GNorm = 2.2509, lr_0 = 6.2203e-04
Validation auc = 0.730047
Epoch 285
Validation auc = 0.736620
Epoch 286
Validation auc = 0.742254
Epoch 287
Validation auc = 0.733803
Epoch 288
Validation auc = 0.727700
Epoch 289
Loss = 2.2580e-01, PNorm = 44.9367, GNorm = 1.7475, lr_0 = 6.1671e-04
Validation auc = 0.723474
Epoch 290
Validation auc = 0.717840
Epoch 291
Validation auc = 0.715493
Epoch 292
Validation auc = 0.711737
Epoch 293
Validation auc = 0.708920
Epoch 294
Loss = 2.3464e-01, PNorm = 45.0167, GNorm = 2.4002, lr_0 = 6.1144e-04
Validation auc = 0.716432
Epoch 295
Validation auc = 0.723944
Epoch 296
Validation auc = 0.730047
Epoch 297
Validation auc = 0.738028
Epoch 298
Validation auc = 0.737559
Epoch 299
Loss = 2.5384e-01, PNorm = 45.1094, GNorm = 1.6387, lr_0 = 6.0621e-04
Validation auc = 0.731455
Model 0 best validation auc = 0.763850 on epoch 40
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.852498
Ensemble test auc = 0.852498
10-fold cross validation
	Seed 0 ==> test auc = 0.856111
	Seed 1 ==> test auc = 0.779049
	Seed 2 ==> test auc = 0.921734
	Seed 3 ==> test auc = 0.797110
	Seed 4 ==> test auc = 0.918724
	Seed 5 ==> test auc = 0.919326
	Seed 6 ==> test auc = 0.913305
	Seed 7 ==> test auc = 0.927152
	Seed 8 ==> test auc = 0.913305
	Seed 9 ==> test auc = 0.852498
Overall test auc = 0.879831 +/- 0.052553
Elapsed time = 0:04:48
Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=1),
 'dropout': 0.30000000000000004,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 300,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 300,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.628169
Epoch 1
Validation auc = 0.695305
Epoch 2
Validation auc = 0.695775
Epoch 3
Validation auc = 0.698592
Epoch 4
Loss = 1.0586e+00, PNorm = 40.6669, GNorm = 12.6522, lr_0 = 6.5000e-04
Validation auc = 0.730986
Epoch 5
Validation auc = 0.719718
Epoch 6
Validation auc = 0.658685
Epoch 7
Validation auc = 0.630986
Epoch 8
Validation auc = 0.614554
Epoch 9
Loss = 9.4697e-01, PNorm = 40.7117, GNorm = 11.4273, lr_0 = 9.9743e-04
Validation auc = 0.592488
Epoch 10
Validation auc = 0.669484
Epoch 11
Validation auc = 0.636150
Epoch 12
Validation auc = 0.654460
Epoch 13
Validation auc = 0.662441
Epoch 14
Loss = 8.9000e-01, PNorm = 40.7700, GNorm = 8.3154, lr_0 = 9.8890e-04
Validation auc = 0.714085
Epoch 15
Validation auc = 0.723474
Epoch 16
Validation auc = 0.698122
Epoch 17
Validation auc = 0.689671
Epoch 18
Validation auc = 0.692488
Epoch 19
Loss = 6.4386e-01, PNorm = 40.8093, GNorm = 5.7968, lr_0 = 9.8045e-04
Validation auc = 0.713146
Epoch 20
Validation auc = 0.737089
Epoch 21
Validation auc = 0.731925
Epoch 22
Validation auc = 0.715493
Epoch 23
Validation auc = 0.715962
Epoch 24
Loss = 7.5890e-01, PNorm = 40.8456, GNorm = 4.7717, lr_0 = 9.7207e-04
Validation auc = 0.718310
Epoch 25
Validation auc = 0.731925
Epoch 26
Validation auc = 0.747887
Epoch 27
Validation auc = 0.760094
Epoch 28
Validation auc = 0.762911
Epoch 29
Loss = 6.4495e-01, PNorm = 40.8848, GNorm = 5.2614, lr_0 = 9.6376e-04
Validation auc = 0.753052
Epoch 30
Validation auc = 0.729577
Epoch 31
Validation auc = 0.701408
Epoch 32
Validation auc = 0.689671
Epoch 33
Validation auc = 0.690610
Epoch 34
Loss = 6.2325e-01, PNorm = 40.9212, GNorm = 6.1661, lr_0 = 9.5552e-04
Validation auc = 0.687793
Epoch 35
Validation auc = 0.690141
Epoch 36
Validation auc = 0.678404
Epoch 37
Validation auc = 0.668545
Epoch 38
Validation auc = 0.644601
Epoch 39
Loss = 5.5793e-01, PNorm = 40.9536, GNorm = 3.7693, lr_0 = 9.4735e-04
Validation auc = 0.650235
Epoch 40
Validation auc = 0.648357
Epoch 41
Validation auc = 0.670423
Epoch 42
Validation auc = 0.678873
Epoch 43
Validation auc = 0.690141
Epoch 44
Loss = 5.1738e-01, PNorm = 40.9941, GNorm = 4.0626, lr_0 = 9.3925e-04
Validation auc = 0.697653
Epoch 45
Validation auc = 0.684038
Epoch 46
Validation auc = 0.674648
Epoch 47
Validation auc = 0.668545
Epoch 48
Validation auc = 0.652113
Epoch 49
Loss = 6.5326e-01, PNorm = 41.0340, GNorm = 5.2204, lr_0 = 9.3122e-04
Validation auc = 0.645540
Epoch 50
Validation auc = 0.654930
Epoch 51
Validation auc = 0.657277
Epoch 52
Validation auc = 0.657277
Epoch 53
Validation auc = 0.651643
Epoch 54
Loss = 5.4901e-01, PNorm = 41.0795, GNorm = 5.6308, lr_0 = 9.2326e-04
Validation auc = 0.646948
Epoch 55
Validation auc = 0.646948
Epoch 56
Validation auc = 0.648826
Epoch 57
Validation auc = 0.649296
Epoch 58
Validation auc = 0.641784
Epoch 59
Loss = 6.5911e-01, PNorm = 41.1254, GNorm = 5.2338, lr_0 = 9.1537e-04
Validation auc = 0.638967
Epoch 60
Validation auc = 0.642254
Epoch 61
Validation auc = 0.638028
Epoch 62
Validation auc = 0.641784
Epoch 63
Validation auc = 0.648357
Epoch 64
Loss = 5.2210e-01, PNorm = 41.1934, GNorm = 4.7278, lr_0 = 9.0754e-04
Validation auc = 0.646479
Epoch 65
Validation auc = 0.641784
Epoch 66
Validation auc = 0.622535
Epoch 67
Validation auc = 0.616901
Epoch 68
Validation auc = 0.618779
Epoch 69
Loss = 4.8703e-01, PNorm = 41.2472, GNorm = 3.2865, lr_0 = 8.9978e-04
Validation auc = 0.628638
Epoch 70
Validation auc = 0.652582
Epoch 71
Validation auc = 0.642723
Epoch 72
Validation auc = 0.647887
Epoch 73
Validation auc = 0.642723
Epoch 74
Loss = 6.0897e-01, PNorm = 41.3447, GNorm = 4.1058, lr_0 = 8.9209e-04
Validation auc = 0.634272
Epoch 75
Validation auc = 0.629108
Epoch 76
Validation auc = 0.628169
Epoch 77
Validation auc = 0.620188
Epoch 78
Validation auc = 0.615962
Epoch 79
Loss = 6.5077e-01, PNorm = 41.4027, GNorm = 4.8542, lr_0 = 8.8447e-04
Validation auc = 0.621596
Epoch 80
Validation auc = 0.618779
Epoch 81
Validation auc = 0.621596
Epoch 82
Validation auc = 0.629108
Epoch 83
Validation auc = 0.631925
Epoch 84
Loss = 4.4870e-01, PNorm = 41.4659, GNorm = 2.9592, lr_0 = 8.7691e-04
Validation auc = 0.630516
Epoch 85
Validation auc = 0.633803
Epoch 86
Validation auc = 0.636150
Epoch 87
Validation auc = 0.635681
Epoch 88
Validation auc = 0.646479
Epoch 89
Loss = 5.3074e-01, PNorm = 41.5356, GNorm = 4.7615, lr_0 = 8.6941e-04
Validation auc = 0.656338
Epoch 90
Validation auc = 0.653052
Epoch 91
Validation auc = 0.650704
Epoch 92
Validation auc = 0.653521
Epoch 93
Validation auc = 0.657277
Epoch 94
Loss = 5.0271e-01, PNorm = 41.5936, GNorm = 2.9955, lr_0 = 8.6198e-04
Validation auc = 0.681221
Epoch 95
Validation auc = 0.676526
Epoch 96
Validation auc = 0.652582
Epoch 97
Validation auc = 0.641315
Epoch 98
Validation auc = 0.642723
Epoch 99
Loss = 4.0967e-01, PNorm = 41.6846, GNorm = 2.3018, lr_0 = 8.5461e-04
Validation auc = 0.651174
Epoch 100
Validation auc = 0.656808
Epoch 101
Validation auc = 0.656338
Epoch 102
Validation auc = 0.665258
Epoch 103
Validation auc = 0.676526
Epoch 104
Loss = 5.1816e-01, PNorm = 41.7601, GNorm = 4.4473, lr_0 = 8.4730e-04
Validation auc = 0.679812
Epoch 105
Validation auc = 0.683099
Epoch 106
Validation auc = 0.672300
Epoch 107
Validation auc = 0.668545
Epoch 108
Validation auc = 0.668075
Epoch 109
Loss = 5.3604e-01, PNorm = 41.8502, GNorm = 2.7722, lr_0 = 8.4006e-04
Validation auc = 0.669484
Epoch 110
Validation auc = 0.665258
Epoch 111
Validation auc = 0.668075
Epoch 112
Validation auc = 0.667606
Epoch 113
Validation auc = 0.666197
Epoch 114
Loss = 4.8188e-01, PNorm = 41.9245, GNorm = 2.7175, lr_0 = 8.3288e-04
Validation auc = 0.670892
Epoch 115
Validation auc = 0.668545
Epoch 116
Validation auc = 0.669484
Epoch 117
Validation auc = 0.670892
Epoch 118
Validation auc = 0.675117
Epoch 119
Loss = 4.4285e-01, PNorm = 42.0460, GNorm = 4.2058, lr_0 = 8.2576e-04
Validation auc = 0.684977
Epoch 120
Validation auc = 0.700000
Epoch 121
Validation auc = 0.707042
Epoch 122
Validation auc = 0.706573
Epoch 123
Validation auc = 0.700939
Epoch 124
Loss = 4.2226e-01, PNorm = 42.1521, GNorm = 3.8331, lr_0 = 8.1870e-04
Validation auc = 0.684977
Epoch 125
Validation auc = 0.676056
Epoch 126
Validation auc = 0.674648
Epoch 127
Validation auc = 0.678873
Epoch 128
Validation auc = 0.672770
Epoch 129
Loss = 3.5606e-01, PNorm = 42.2504, GNorm = 1.9712, lr_0 = 8.1170e-04
Validation auc = 0.659155
Epoch 130
Validation auc = 0.666197
Epoch 131
Validation auc = 0.673709
Epoch 132
Validation auc = 0.673709
Epoch 133
Validation auc = 0.666197
Epoch 134
Loss = 4.2385e-01, PNorm = 42.3388, GNorm = 3.3083, lr_0 = 8.0476e-04
Validation auc = 0.653521
Epoch 135
Validation auc = 0.656338
Epoch 136
Validation auc = 0.672770
Epoch 137
Validation auc = 0.689202
Epoch 138
Validation auc = 0.688263
Epoch 139
Loss = 3.8030e-01, PNorm = 42.4286, GNorm = 2.9730, lr_0 = 7.9788e-04
Validation auc = 0.687324
Epoch 140
Validation auc = 0.697653
Epoch 141
Validation auc = 0.707512
Epoch 142
Validation auc = 0.718310
Epoch 143
Validation auc = 0.723944
Epoch 144
Loss = 4.4222e-01, PNorm = 42.5211, GNorm = 2.4797, lr_0 = 7.9106e-04
Validation auc = 0.722535
Epoch 145
Validation auc = 0.716901
Epoch 146
Validation auc = 0.715023
Epoch 147
Validation auc = 0.717371
Epoch 148
Validation auc = 0.716901
Epoch 149
Loss = 3.7644e-01, PNorm = 42.6240, GNorm = 3.6333, lr_0 = 7.8430e-04
Validation auc = 0.711268
Epoch 150
Validation auc = 0.717371
Epoch 151
Validation auc = 0.723474
Epoch 152
Validation auc = 0.728169
Epoch 153
Validation auc = 0.736150
Epoch 154
Loss = 4.3723e-01, PNorm = 42.7339, GNorm = 2.5887, lr_0 = 7.7759e-04
Validation auc = 0.749765
Epoch 155
Validation auc = 0.746948
Epoch 156
Validation auc = 0.740376
Epoch 157
Validation auc = 0.736150
Epoch 158
Validation auc = 0.726761
Epoch 159
Loss = 3.6100e-01, PNorm = 42.8272, GNorm = 3.5246, lr_0 = 7.7095e-04
Validation auc = 0.722066
Epoch 160
Validation auc = 0.715023
Epoch 161
Validation auc = 0.711737
Epoch 162
Validation auc = 0.722535
Epoch 163
Validation auc = 0.727230
Epoch 164
Loss = 4.1562e-01, PNorm = 42.9251, GNorm = 2.8812, lr_0 = 7.6436e-04
Validation auc = 0.731455
Epoch 165
Validation auc = 0.735681
Epoch 166
Validation auc = 0.729577
Epoch 167
Validation auc = 0.715493
Epoch 168
Validation auc = 0.701878
Epoch 169
Loss = 3.3769e-01, PNorm = 43.0134, GNorm = 3.8247, lr_0 = 7.5782e-04
Validation auc = 0.699531
Epoch 170
Validation auc = 0.697653
Epoch 171
Validation auc = 0.703286
Epoch 172
Validation auc = 0.700939
Epoch 173
Validation auc = 0.704225
Epoch 174
Loss = 3.7853e-01, PNorm = 43.1116, GNorm = 2.9030, lr_0 = 7.5134e-04
Validation auc = 0.715023
Epoch 175
Validation auc = 0.716901
Epoch 176
Validation auc = 0.728638
Epoch 177
Validation auc = 0.728638
Epoch 178
Validation auc = 0.727700
Epoch 179
Loss = 3.6827e-01, PNorm = 43.2310, GNorm = 2.7030, lr_0 = 7.4492e-04
Validation auc = 0.729577
Epoch 180
Validation auc = 0.729577
Epoch 181
Validation auc = 0.710798
Epoch 182
Validation auc = 0.704695
Epoch 183
Validation auc = 0.704225
Epoch 184
Loss = 3.2601e-01, PNorm = 43.3343, GNorm = 2.4061, lr_0 = 7.3855e-04
Validation auc = 0.709859
Epoch 185
Validation auc = 0.715023
Epoch 186
Validation auc = 0.715493
Epoch 187
Validation auc = 0.711268
Epoch 188
Validation auc = 0.712207
Epoch 189
Loss = 4.9626e-01, PNorm = 43.4414, GNorm = 4.7753, lr_0 = 7.3224e-04
Validation auc = 0.719249
Epoch 190
Validation auc = 0.730516
Epoch 191
Validation auc = 0.730986
Epoch 192
Validation auc = 0.726761
Epoch 193
Validation auc = 0.723944
Epoch 194
Loss = 3.4785e-01, PNorm = 43.5355, GNorm = 2.0181, lr_0 = 7.2598e-04
Validation auc = 0.723005
Epoch 195
Validation auc = 0.734272
Epoch 196
Validation auc = 0.734272
Epoch 197
Validation auc = 0.731455
Epoch 198
Validation auc = 0.733333
Epoch 199
Loss = 2.9465e-01, PNorm = 43.6278, GNorm = 2.5281, lr_0 = 7.1977e-04
Validation auc = 0.729577
Epoch 200
Validation auc = 0.723474
Epoch 201
Validation auc = 0.728638
Epoch 202
Validation auc = 0.730516
Epoch 203
Validation auc = 0.730047
Epoch 204
Loss = 3.0448e-01, PNorm = 43.7115, GNorm = 2.9157, lr_0 = 7.1362e-04
Validation auc = 0.728169
Epoch 205
Validation auc = 0.727700
Epoch 206
Validation auc = 0.729108
Epoch 207
Validation auc = 0.730047
Epoch 208
Validation auc = 0.735681
Epoch 209
Loss = 2.1663e-01, PNorm = 43.8098, GNorm = 2.9022, lr_0 = 7.0752e-04
Validation auc = 0.734742
Epoch 210
Validation auc = 0.734272
Epoch 211
Validation auc = 0.730516
Epoch 212
Validation auc = 0.730516
Epoch 213
Validation auc = 0.730047
Epoch 214
Loss = 3.1711e-01, PNorm = 43.9030, GNorm = 3.3697, lr_0 = 7.0147e-04
Validation auc = 0.729108
Epoch 215
Validation auc = 0.727230
Epoch 216
Validation auc = 0.720188
Epoch 217
Validation auc = 0.715493
Epoch 218
Validation auc = 0.715962
Epoch 219
Loss = 2.7173e-01, PNorm = 43.9853, GNorm = 2.8156, lr_0 = 6.9548e-04
Validation auc = 0.718310
Epoch 220
Validation auc = 0.717840
Epoch 221
Validation auc = 0.720657
Epoch 222
Validation auc = 0.721596
Epoch 223
Validation auc = 0.727230
Epoch 224
Loss = 2.6804e-01, PNorm = 44.0681, GNorm = 3.0562, lr_0 = 6.8953e-04
Validation auc = 0.730047
Epoch 225
Validation auc = 0.736150
Epoch 226
Validation auc = 0.735211
Epoch 227
Validation auc = 0.731455
Epoch 228
Validation auc = 0.723474
Epoch 229
Loss = 3.6117e-01, PNorm = 44.1421, GNorm = 2.2142, lr_0 = 6.8364e-04
Validation auc = 0.730986
Epoch 230
Validation auc = 0.729108
Epoch 231
Validation auc = 0.727230
Epoch 232
Validation auc = 0.725352
Epoch 233
Validation auc = 0.724883
Epoch 234
Loss = 2.2079e-01, PNorm = 44.2325, GNorm = 1.7093, lr_0 = 6.7779e-04
Validation auc = 0.723944
Epoch 235
Validation auc = 0.725822
Epoch 236
Validation auc = 0.723944
Epoch 237
Validation auc = 0.728169
Epoch 238
Validation auc = 0.729108
Epoch 239
Loss = 2.8374e-01, PNorm = 44.3233, GNorm = 1.9163, lr_0 = 6.7200e-04
Validation auc = 0.728638
Epoch 240
Validation auc = 0.733333
Epoch 241
Validation auc = 0.736620
Epoch 242
Validation auc = 0.743662
Epoch 243
Validation auc = 0.740376
Epoch 244
Loss = 2.6964e-01, PNorm = 44.4131, GNorm = 1.9107, lr_0 = 6.6625e-04
Validation auc = 0.740376
Epoch 245
Validation auc = 0.746479
Epoch 246
Validation auc = 0.750704
Epoch 247
Validation auc = 0.750235
Epoch 248
Validation auc = 0.746948
Epoch 249
Loss = 3.0184e-01, PNorm = 44.4966, GNorm = 2.6694, lr_0 = 6.6056e-04
Validation auc = 0.739906
Epoch 250
Validation auc = 0.735681
Epoch 251
Validation auc = 0.733333
Epoch 252
Validation auc = 0.731455
Epoch 253
Validation auc = 0.730516
Epoch 254
Loss = 2.6537e-01, PNorm = 44.5765, GNorm = 2.5545, lr_0 = 6.5491e-04
Validation auc = 0.730047
Epoch 255
Validation auc = 0.731455
Epoch 256
Validation auc = 0.732394
Epoch 257
Validation auc = 0.733803
Epoch 258
Validation auc = 0.734272
Epoch 259
Loss = 2.5517e-01, PNorm = 44.6631, GNorm = 1.3606, lr_0 = 6.4931e-04
Validation auc = 0.735681
Epoch 260
Validation auc = 0.733333
Epoch 261
Validation auc = 0.729108
Epoch 262
Validation auc = 0.723474
Epoch 263
Validation auc = 0.717840
Epoch 264
Loss = 3.0369e-01, PNorm = 44.7385, GNorm = 2.0090, lr_0 = 6.4376e-04
Validation auc = 0.715962
Epoch 265
Validation auc = 0.714085
Epoch 266
Validation auc = 0.713615
Epoch 267
Validation auc = 0.715493
Epoch 268
Validation auc = 0.717840
Epoch 269
Loss = 2.3530e-01, PNorm = 44.8324, GNorm = 2.5031, lr_0 = 6.3826e-04
Validation auc = 0.717840
Epoch 270
Validation auc = 0.721596
Epoch 271
Validation auc = 0.722535
Epoch 272
Validation auc = 0.724413
Epoch 273
Validation auc = 0.726761
Epoch 274
Loss = 1.9287e-01, PNorm = 44.9034, GNorm = 1.1133, lr_0 = 6.3280e-04
Validation auc = 0.728638
Epoch 275
Validation auc = 0.726761
Epoch 276
Validation auc = 0.722535
Epoch 277
Validation auc = 0.720188
Epoch 278
Validation auc = 0.723944
Epoch 279
Loss = 2.6623e-01, PNorm = 44.9745, GNorm = 3.6322, lr_0 = 6.2739e-04
Validation auc = 0.723005
Epoch 280
Validation auc = 0.719249
Epoch 281
Validation auc = 0.715962
Epoch 282
Validation auc = 0.713146
Epoch 283
Validation auc = 0.715962
Epoch 284
Loss = 2.0346e-01, PNorm = 45.0538, GNorm = 3.4675, lr_0 = 6.2203e-04
Validation auc = 0.715493
Epoch 285
Validation auc = 0.723005
Epoch 286
Validation auc = 0.728169
Epoch 287
Validation auc = 0.721127
Epoch 288
Validation auc = 0.717371
Epoch 289
Loss = 2.7355e-01, PNorm = 45.1216, GNorm = 3.2732, lr_0 = 6.1671e-04
Validation auc = 0.713146
Epoch 290
Validation auc = 0.714085
Epoch 291
Validation auc = 0.707042
Epoch 292
Validation auc = 0.712207
Epoch 293
Validation auc = 0.710329
Epoch 294
Loss = 2.7110e-01, PNorm = 45.2007, GNorm = 4.1903, lr_0 = 6.1144e-04
Validation auc = 0.708451
Epoch 295
Validation auc = 0.709859
Epoch 296
Validation auc = 0.720657
Epoch 297
Validation auc = 0.723474
Epoch 298
Validation auc = 0.725352
Epoch 299
Loss = 2.2368e-01, PNorm = 45.3015, GNorm = 1.8506, lr_0 = 6.0621e-04
Validation auc = 0.722535
Model 0 best validation auc = 0.762911 on epoch 28
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.856111
Ensemble test auc = 0.856111
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.649765
Epoch 1
Validation auc = 0.674178
Epoch 2
Validation auc = 0.667606
Epoch 3
Validation auc = 0.668545
Epoch 4
Loss = 1.0876e+00, PNorm = 40.6664, GNorm = 15.9575, lr_0 = 6.5000e-04
Validation auc = 0.664789
Epoch 5
Validation auc = 0.671362
Epoch 6
Validation auc = 0.660563
Epoch 7
Validation auc = 0.659155
Epoch 8
Validation auc = 0.669484
Epoch 9
Loss = 1.0050e+00, PNorm = 40.7127, GNorm = 9.4112, lr_0 = 9.9743e-04
Validation auc = 0.670423
Epoch 10
Validation auc = 0.734742
Epoch 11
Validation auc = 0.642723
Epoch 12
Validation auc = 0.659624
Epoch 13
Validation auc = 0.672300
Epoch 14
Loss = 7.5446e-01, PNorm = 40.7584, GNorm = 7.9101, lr_0 = 9.8890e-04
Validation auc = 0.732394
Epoch 15
Validation auc = 0.777934
Epoch 16
Validation auc = 0.690610
Epoch 17
Validation auc = 0.690610
Epoch 18
Validation auc = 0.705634
Epoch 19
Loss = 6.8242e-01, PNorm = 40.7987, GNorm = 6.0677, lr_0 = 9.8045e-04
Validation auc = 0.683568
Epoch 20
Validation auc = 0.663380
Epoch 21
Validation auc = 0.658685
Epoch 22
Validation auc = 0.661972
Epoch 23
Validation auc = 0.666197
Epoch 24
Loss = 6.5581e-01, PNorm = 40.8466, GNorm = 4.5116, lr_0 = 9.7207e-04
Validation auc = 0.665258
Epoch 25
Validation auc = 0.664789
Epoch 26
Validation auc = 0.668545
Epoch 27
Validation auc = 0.682160
Epoch 28
Validation auc = 0.699061
Epoch 29
Loss = 5.6465e-01, PNorm = 40.8680, GNorm = 4.2024, lr_0 = 9.6376e-04
Validation auc = 0.712676
Epoch 30
Validation auc = 0.706573
Epoch 31
Validation auc = 0.697653
Epoch 32
Validation auc = 0.686385
Epoch 33
Validation auc = 0.666667
Epoch 34
Loss = 5.9160e-01, PNorm = 40.9107, GNorm = 9.3277, lr_0 = 9.5552e-04
Validation auc = 0.640845
Epoch 35
Validation auc = 0.631455
Epoch 36
Validation auc = 0.620657
Epoch 37
Validation auc = 0.632394
Epoch 38
Validation auc = 0.625822
Epoch 39
Loss = 6.4413e-01, PNorm = 40.9579, GNorm = 4.8620, lr_0 = 9.4735e-04
Validation auc = 0.621127
Epoch 40
Validation auc = 0.617371
Epoch 41
Validation auc = 0.618310
Epoch 42
Validation auc = 0.614554
Epoch 43
Validation auc = 0.613146
Epoch 44
Loss = 5.8825e-01, PNorm = 40.9844, GNorm = 3.8137, lr_0 = 9.3925e-04
Validation auc = 0.614085
Epoch 45
Validation auc = 0.631455
Epoch 46
Validation auc = 0.635211
Epoch 47
Validation auc = 0.625822
Epoch 48
Validation auc = 0.632864
Epoch 49
Loss = 5.8276e-01, PNorm = 41.0387, GNorm = 4.7174, lr_0 = 9.3122e-04
Validation auc = 0.635681
Epoch 50
Validation auc = 0.620657
Epoch 51
Validation auc = 0.626761
Epoch 52
Validation auc = 0.658685
Epoch 53
Validation auc = 0.652582
Epoch 54
Loss = 5.7388e-01, PNorm = 41.0799, GNorm = 2.5862, lr_0 = 9.2326e-04
Validation auc = 0.643192
Epoch 55
Validation auc = 0.643192
Epoch 56
Validation auc = 0.639437
Epoch 57
Validation auc = 0.649765
Epoch 58
Validation auc = 0.672770
Epoch 59
Loss = 5.8007e-01, PNorm = 41.1300, GNorm = 5.0977, lr_0 = 9.1537e-04
Validation auc = 0.650704
Epoch 60
Validation auc = 0.637559
Epoch 61
Validation auc = 0.661502
Epoch 62
Validation auc = 0.662911
Epoch 63
Validation auc = 0.657746
Epoch 64
Loss = 5.7533e-01, PNorm = 41.1737, GNorm = 4.6470, lr_0 = 9.0754e-04
Validation auc = 0.653052
Epoch 65
Validation auc = 0.655399
Epoch 66
Validation auc = 0.668075
Epoch 67
Validation auc = 0.665258
Epoch 68
Validation auc = 0.645540
Epoch 69
Loss = 5.4912e-01, PNorm = 41.2251, GNorm = 3.4596, lr_0 = 8.9978e-04
Validation auc = 0.631925
Epoch 70
Validation auc = 0.637089
Epoch 71
Validation auc = 0.642723
Epoch 72
Validation auc = 0.640376
Epoch 73
Validation auc = 0.643192
Epoch 74
Loss = 5.6889e-01, PNorm = 41.2884, GNorm = 3.3854, lr_0 = 8.9209e-04
Validation auc = 0.638028
Epoch 75
Validation auc = 0.638967
Epoch 76
Validation auc = 0.634742
Epoch 77
Validation auc = 0.636150
Epoch 78
Validation auc = 0.645070
Epoch 79
Loss = 5.4145e-01, PNorm = 41.3278, GNorm = 2.9870, lr_0 = 8.8447e-04
Validation auc = 0.645070
Epoch 80
Validation auc = 0.652582
Epoch 81
Validation auc = 0.648826
Epoch 82
Validation auc = 0.640376
Epoch 83
Validation auc = 0.635211
Epoch 84
Loss = 4.8474e-01, PNorm = 41.3881, GNorm = 3.2753, lr_0 = 8.7691e-04
Validation auc = 0.644131
Epoch 85
Validation auc = 0.643662
Epoch 86
Validation auc = 0.645540
Epoch 87
Validation auc = 0.653991
Epoch 88
Validation auc = 0.647887
Epoch 89
Loss = 4.7768e-01, PNorm = 41.4428, GNorm = 2.8218, lr_0 = 8.6941e-04
Validation auc = 0.647418
Epoch 90
Validation auc = 0.646948
Epoch 91
Validation auc = 0.664789
Epoch 92
Validation auc = 0.700939
Epoch 93
Validation auc = 0.701878
Epoch 94
Loss = 5.0753e-01, PNorm = 41.5239, GNorm = 2.3757, lr_0 = 8.6198e-04
Validation auc = 0.696244
Epoch 95
Validation auc = 0.680751
Epoch 96
Validation auc = 0.669484
Epoch 97
Validation auc = 0.661972
Epoch 98
Validation auc = 0.671362
Epoch 99
Loss = 4.0298e-01, PNorm = 41.6022, GNorm = 2.5264, lr_0 = 8.5461e-04
Validation auc = 0.672770
Epoch 100
Validation auc = 0.675117
Epoch 101
Validation auc = 0.675117
Epoch 102
Validation auc = 0.666667
Epoch 103
Validation auc = 0.662441
Epoch 104
Loss = 5.3391e-01, PNorm = 41.6816, GNorm = 3.8458, lr_0 = 8.4730e-04
Validation auc = 0.659624
Epoch 105
Validation auc = 0.661502
Epoch 106
Validation auc = 0.662911
Epoch 107
Validation auc = 0.670892
Epoch 108
Validation auc = 0.684038
Epoch 109
Loss = 4.4263e-01, PNorm = 41.7511, GNorm = 2.5768, lr_0 = 8.4006e-04
Validation auc = 0.695305
Epoch 110
Validation auc = 0.704695
Epoch 111
Validation auc = 0.707512
Epoch 112
Validation auc = 0.708451
Epoch 113
Validation auc = 0.706573
Epoch 114
Loss = 4.4434e-01, PNorm = 41.8346, GNorm = 2.3888, lr_0 = 8.3288e-04
Validation auc = 0.700939
Epoch 115
Validation auc = 0.692488
Epoch 116
Validation auc = 0.686854
Epoch 117
Validation auc = 0.686854
Epoch 118
Validation auc = 0.696714
Epoch 119
Loss = 4.9984e-01, PNorm = 41.9289, GNorm = 2.5198, lr_0 = 8.2576e-04
Validation auc = 0.712676
Epoch 120
Validation auc = 0.728169
Epoch 121
Validation auc = 0.732394
Epoch 122
Validation auc = 0.727700
Epoch 123
Validation auc = 0.721596
Epoch 124
Loss = 4.4344e-01, PNorm = 42.0328, GNorm = 2.4286, lr_0 = 8.1870e-04
Validation auc = 0.711737
Epoch 125
Validation auc = 0.712207
Epoch 126
Validation auc = 0.710798
Epoch 127
Validation auc = 0.698122
Epoch 128
Validation auc = 0.702817
Epoch 129
Loss = 5.1110e-01, PNorm = 42.1331, GNorm = 3.1030, lr_0 = 8.1170e-04
Validation auc = 0.705164
Epoch 130
Validation auc = 0.704225
Epoch 131
Validation auc = 0.712207
Epoch 132
Validation auc = 0.719249
Epoch 133
Validation auc = 0.717371
Epoch 134
Loss = 4.0428e-01, PNorm = 42.2440, GNorm = 2.0037, lr_0 = 8.0476e-04
Validation auc = 0.722066
Epoch 135
Validation auc = 0.720188
Epoch 136
Validation auc = 0.707981
Epoch 137
Validation auc = 0.702347
Epoch 138
Validation auc = 0.699531
Epoch 139
Loss = 3.4503e-01, PNorm = 42.3450, GNorm = 2.4817, lr_0 = 7.9788e-04
Validation auc = 0.700939
Epoch 140
Validation auc = 0.709390
Epoch 141
Validation auc = 0.715493
Epoch 142
Validation auc = 0.716432
Epoch 143
Validation auc = 0.721127
Epoch 144
Loss = 3.9864e-01, PNorm = 42.4499, GNorm = 2.7479, lr_0 = 7.9106e-04
Validation auc = 0.723474
Epoch 145
Validation auc = 0.729577
Epoch 146
Validation auc = 0.737089
Epoch 147
Validation auc = 0.746479
Epoch 148
Validation auc = 0.747418
Epoch 149
Loss = 4.3380e-01, PNorm = 42.5534, GNorm = 3.7717, lr_0 = 7.8430e-04
Validation auc = 0.738028
Epoch 150
Validation auc = 0.744601
Epoch 151
Validation auc = 0.737559
Epoch 152
Validation auc = 0.726761
Epoch 153
Validation auc = 0.733333
Epoch 154
Loss = 3.4926e-01, PNorm = 42.6534, GNorm = 2.7665, lr_0 = 7.7759e-04
Validation auc = 0.732394
Epoch 155
Validation auc = 0.748357
Epoch 156
Validation auc = 0.756338
Epoch 157
Validation auc = 0.756808
Epoch 158
Validation auc = 0.751643
Epoch 159
Loss = 3.6341e-01, PNorm = 42.7375, GNorm = 2.2139, lr_0 = 7.7095e-04
Validation auc = 0.746479
Epoch 160
Validation auc = 0.755869
Epoch 161
Validation auc = 0.760563
Epoch 162
Validation auc = 0.755399
Epoch 163
Validation auc = 0.758685
Epoch 164
Loss = 3.8049e-01, PNorm = 42.8171, GNorm = 2.4252, lr_0 = 7.6436e-04
Validation auc = 0.755399
Epoch 165
Validation auc = 0.749765
Epoch 166
Validation auc = 0.748826
Epoch 167
Validation auc = 0.746009
Epoch 168
Validation auc = 0.724883
Epoch 169
Loss = 3.8821e-01, PNorm = 42.9019, GNorm = 1.6559, lr_0 = 7.5782e-04
Validation auc = 0.715493
Epoch 170
Validation auc = 0.717371
Epoch 171
Validation auc = 0.715023
Epoch 172
Validation auc = 0.717371
Epoch 173
Validation auc = 0.720657
Epoch 174
Loss = 3.8514e-01, PNorm = 43.0044, GNorm = 2.9102, lr_0 = 7.5134e-04
Validation auc = 0.724883
Epoch 175
Validation auc = 0.730047
Epoch 176
Validation auc = 0.730986
Epoch 177
Validation auc = 0.729577
Epoch 178
Validation auc = 0.734272
Epoch 179
Loss = 3.1922e-01, PNorm = 43.0983, GNorm = 2.4689, lr_0 = 7.4492e-04
Validation auc = 0.730986
Epoch 180
Validation auc = 0.730047
Epoch 181
Validation auc = 0.721596
Epoch 182
Validation auc = 0.735681
Epoch 183
Validation auc = 0.750235
Epoch 184
Loss = 3.7262e-01, PNorm = 43.1781, GNorm = 2.8077, lr_0 = 7.3855e-04
Validation auc = 0.761033
Epoch 185
Validation auc = 0.761033
Epoch 186
Validation auc = 0.758685
Epoch 187
Validation auc = 0.756338
Epoch 188
Validation auc = 0.750704
Epoch 189
Loss = 5.2751e-01, PNorm = 43.2531, GNorm = 4.5642, lr_0 = 7.3224e-04
Validation auc = 0.749296
Epoch 190
Validation auc = 0.746479
Epoch 191
Validation auc = 0.749296
Epoch 192
Validation auc = 0.744601
Epoch 193
Validation auc = 0.733803
Epoch 194
Loss = 3.9580e-01, PNorm = 43.3283, GNorm = 3.3242, lr_0 = 7.2598e-04
Validation auc = 0.738028
Epoch 195
Validation auc = 0.744131
Epoch 196
Validation auc = 0.753521
Epoch 197
Validation auc = 0.756808
Epoch 198
Validation auc = 0.756808
Epoch 199
Loss = 3.0891e-01, PNorm = 43.4174, GNorm = 1.9664, lr_0 = 7.1977e-04
Validation auc = 0.760094
Epoch 200
Validation auc = 0.755869
Epoch 201
Validation auc = 0.753052
Epoch 202
Validation auc = 0.753991
Epoch 203
Validation auc = 0.750704
Epoch 204
Loss = 2.8270e-01, PNorm = 43.5025, GNorm = 2.6705, lr_0 = 7.1362e-04
Validation auc = 0.747418
Epoch 205
Validation auc = 0.741315
Epoch 206
Validation auc = 0.751174
Epoch 207
Validation auc = 0.744131
Epoch 208
Validation auc = 0.739906
Epoch 209
Loss = 3.6347e-01, PNorm = 43.5936, GNorm = 3.9375, lr_0 = 7.0752e-04
Validation auc = 0.738028
Epoch 210
Validation auc = 0.733333
Epoch 211
Validation auc = 0.735681
Epoch 212
Validation auc = 0.742723
Epoch 213
Validation auc = 0.750704
Epoch 214
Loss = 2.4105e-01, PNorm = 43.6841, GNorm = 1.9146, lr_0 = 7.0147e-04
Validation auc = 0.756338
Epoch 215
Validation auc = 0.760563
Epoch 216
Validation auc = 0.763380
Epoch 217
Validation auc = 0.761033
Epoch 218
Validation auc = 0.754460
Epoch 219
Loss = 3.0605e-01, PNorm = 43.7635, GNorm = 2.8301, lr_0 = 6.9548e-04
Validation auc = 0.753052
Epoch 220
Validation auc = 0.754460
Epoch 221
Validation auc = 0.746479
Epoch 222
Validation auc = 0.746479
Epoch 223
Validation auc = 0.740845
Epoch 224
Loss = 3.8233e-01, PNorm = 43.8460, GNorm = 3.8259, lr_0 = 6.8953e-04
Validation auc = 0.733803
Epoch 225
Validation auc = 0.718779
Epoch 226
Validation auc = 0.718310
Epoch 227
Validation auc = 0.712676
Epoch 228
Validation auc = 0.713146
Epoch 229
Loss = 2.6902e-01, PNorm = 43.9145, GNorm = 2.7136, lr_0 = 6.8364e-04
Validation auc = 0.713615
Epoch 230
Validation auc = 0.711737
Epoch 231
Validation auc = 0.718779
Epoch 232
Validation auc = 0.728638
Epoch 233
Validation auc = 0.737559
Epoch 234
Loss = 2.9793e-01, PNorm = 44.0015, GNorm = 2.6566, lr_0 = 6.7779e-04
Validation auc = 0.741784
Epoch 235
Validation auc = 0.741784
Epoch 236
Validation auc = 0.742723
Epoch 237
Validation auc = 0.740845
Epoch 238
Validation auc = 0.748826
Epoch 239
Loss = 2.9457e-01, PNorm = 44.0980, GNorm = 2.9897, lr_0 = 6.7200e-04
Validation auc = 0.755399
Epoch 240
Validation auc = 0.752582
Epoch 241
Validation auc = 0.738498
Epoch 242
Validation auc = 0.732864
Epoch 243
Validation auc = 0.728638
Epoch 244
Loss = 3.8543e-01, PNorm = 44.1730, GNorm = 4.5694, lr_0 = 6.6625e-04
Validation auc = 0.726761
Epoch 245
Validation auc = 0.733333
Epoch 246
Validation auc = 0.733333
Epoch 247
Validation auc = 0.738967
Epoch 248
Validation auc = 0.743662
Epoch 249
Loss = 3.9622e-01, PNorm = 44.2576, GNorm = 7.8824, lr_0 = 6.6056e-04
Validation auc = 0.742723
Epoch 250
Validation auc = 0.731455
Epoch 251
Validation auc = 0.715023
Epoch 252
Validation auc = 0.709859
Epoch 253
Validation auc = 0.714085
Epoch 254
Loss = 3.5730e-01, PNorm = 44.3315, GNorm = 2.6143, lr_0 = 6.5491e-04
Validation auc = 0.714085
Epoch 255
Validation auc = 0.715493
Epoch 256
Validation auc = 0.716432
Epoch 257
Validation auc = 0.721596
Epoch 258
Validation auc = 0.720188
Epoch 259
Loss = 3.4132e-01, PNorm = 44.3949, GNorm = 5.0619, lr_0 = 6.4931e-04
Validation auc = 0.722066
Epoch 260
Validation auc = 0.730047
Epoch 261
Validation auc = 0.730047
Epoch 262
Validation auc = 0.732394
Epoch 263
Validation auc = 0.729108
Epoch 264
Loss = 3.3519e-01, PNorm = 44.4651, GNorm = 4.9413, lr_0 = 6.4376e-04
Validation auc = 0.730986
Epoch 265
Validation auc = 0.733333
Epoch 266
Validation auc = 0.729108
Epoch 267
Validation auc = 0.729577
Epoch 268
Validation auc = 0.725822
Epoch 269
Loss = 2.8320e-01, PNorm = 44.5310, GNorm = 1.9137, lr_0 = 6.3826e-04
Validation auc = 0.730047
Epoch 270
Validation auc = 0.729577
Epoch 271
Validation auc = 0.727230
Epoch 272
Validation auc = 0.735681
Epoch 273
Validation auc = 0.741315
Epoch 274
Loss = 2.5834e-01, PNorm = 44.5905, GNorm = 2.5606, lr_0 = 6.3280e-04
Validation auc = 0.741315
Epoch 275
Validation auc = 0.737089
Epoch 276
Validation auc = 0.736620
Epoch 277
Validation auc = 0.730986
Epoch 278
Validation auc = 0.732394
Epoch 279
Loss = 2.2140e-01, PNorm = 44.6544, GNorm = 1.7767, lr_0 = 6.2739e-04
Validation auc = 0.731925
Epoch 280
Validation auc = 0.734272
Epoch 281
Validation auc = 0.734272
Epoch 282
Validation auc = 0.731925
Epoch 283
Validation auc = 0.731925
Epoch 284
Loss = 2.2407e-01, PNorm = 44.7271, GNorm = 2.9690, lr_0 = 6.2203e-04
Validation auc = 0.734272
Epoch 285
Validation auc = 0.736150
Epoch 286
Validation auc = 0.737089
Epoch 287
Validation auc = 0.743192
Epoch 288
Validation auc = 0.737559
Epoch 289
Loss = 2.5983e-01, PNorm = 44.7923, GNorm = 2.3640, lr_0 = 6.1671e-04
Validation auc = 0.734742
Epoch 290
Validation auc = 0.731455
Epoch 291
Validation auc = 0.726761
Epoch 292
Validation auc = 0.721596
Epoch 293
Validation auc = 0.718779
Epoch 294
Loss = 2.5323e-01, PNorm = 44.8473, GNorm = 2.1431, lr_0 = 6.1144e-04
Validation auc = 0.718310
Epoch 295
Validation auc = 0.724413
Epoch 296
Validation auc = 0.726761
Epoch 297
Validation auc = 0.730047
Epoch 298
Validation auc = 0.735211
Epoch 299
Loss = 2.3729e-01, PNorm = 44.9183, GNorm = 2.0203, lr_0 = 6.0621e-04
Validation auc = 0.738967
Model 0 best validation auc = 0.777934 on epoch 15
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.779049
Ensemble test auc = 0.779049
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.661972
Epoch 1
Validation auc = 0.682160
Epoch 2
Validation auc = 0.680751
Epoch 3
Validation auc = 0.684507
Epoch 4
Loss = 1.1781e+00, PNorm = 40.6682, GNorm = 15.0723, lr_0 = 6.5000e-04
Validation auc = 0.689671
Epoch 5
Validation auc = 0.723474
Epoch 6
Validation auc = 0.690610
Epoch 7
Validation auc = 0.663380
Epoch 8
Validation auc = 0.659624
Epoch 9
Loss = 9.1711e-01, PNorm = 40.7127, GNorm = 7.3795, lr_0 = 9.9743e-04
Validation auc = 0.662911
Epoch 10
Validation auc = 0.651174
Epoch 11
Validation auc = 0.634742
Epoch 12
Validation auc = 0.628638
Epoch 13
Validation auc = 0.628169
Epoch 14
Loss = 7.2090e-01, PNorm = 40.7645, GNorm = 9.7656, lr_0 = 9.8890e-04
Validation auc = 0.651174
Epoch 15
Validation auc = 0.661502
Epoch 16
Validation auc = 0.652113
Epoch 17
Validation auc = 0.659624
Epoch 18
Validation auc = 0.669953
Epoch 19
Loss = 8.5540e-01, PNorm = 40.8080, GNorm = 7.9993, lr_0 = 9.8045e-04
Validation auc = 0.637559
Epoch 20
Validation auc = 0.624883
Epoch 21
Validation auc = 0.628169
Epoch 22
Validation auc = 0.634272
Epoch 23
Validation auc = 0.636150
Epoch 24
Loss = 6.2496e-01, PNorm = 40.8435, GNorm = 6.3456, lr_0 = 9.7207e-04
Validation auc = 0.643662
Epoch 25
Validation auc = 0.647887
Epoch 26
Validation auc = 0.653052
Epoch 27
Validation auc = 0.651643
Epoch 28
Validation auc = 0.644601
Epoch 29
Loss = 7.7754e-01, PNorm = 40.8808, GNorm = 5.8928, lr_0 = 9.6376e-04
Validation auc = 0.630986
Epoch 30
Validation auc = 0.643662
Epoch 31
Validation auc = 0.648357
Epoch 32
Validation auc = 0.630986
Epoch 33
Validation auc = 0.633803
Epoch 34
Loss = 6.5506e-01, PNorm = 40.9031, GNorm = 3.4445, lr_0 = 9.5552e-04
Validation auc = 0.630986
Epoch 35
Validation auc = 0.641315
Epoch 36
Validation auc = 0.641315
Epoch 37
Validation auc = 0.643192
Epoch 38
Validation auc = 0.650235
Epoch 39
Loss = 6.6351e-01, PNorm = 40.9500, GNorm = 5.1138, lr_0 = 9.4735e-04
Validation auc = 0.656338
Epoch 40
Validation auc = 0.653991
Epoch 41
Validation auc = 0.645540
Epoch 42
Validation auc = 0.643192
Epoch 43
Validation auc = 0.648826
Epoch 44
Loss = 5.9955e-01, PNorm = 40.9732, GNorm = 4.3191, lr_0 = 9.3925e-04
Validation auc = 0.638498
Epoch 45
Validation auc = 0.631925
Epoch 46
Validation auc = 0.623005
Epoch 47
Validation auc = 0.628169
Epoch 48
Validation auc = 0.626761
Epoch 49
Loss = 6.6602e-01, PNorm = 41.0217, GNorm = 3.8606, lr_0 = 9.3122e-04
Validation auc = 0.620657
Epoch 50
Validation auc = 0.616432
Epoch 51
Validation auc = 0.622066
Epoch 52
Validation auc = 0.633803
Epoch 53
Validation auc = 0.640376
Epoch 54
Loss = 5.4955e-01, PNorm = 41.0629, GNorm = 4.5238, lr_0 = 9.2326e-04
Validation auc = 0.633803
Epoch 55
Validation auc = 0.647887
Epoch 56
Validation auc = 0.655869
Epoch 57
Validation auc = 0.667136
Epoch 58
Validation auc = 0.675117
Epoch 59
Loss = 5.9369e-01, PNorm = 41.1049, GNorm = 4.1474, lr_0 = 9.1537e-04
Validation auc = 0.671831
Epoch 60
Validation auc = 0.663850
Epoch 61
Validation auc = 0.662441
Epoch 62
Validation auc = 0.657746
Epoch 63
Validation auc = 0.648357
Epoch 64
Loss = 5.3629e-01, PNorm = 41.1613, GNorm = 3.8278, lr_0 = 9.0754e-04
Validation auc = 0.640376
Epoch 65
Validation auc = 0.648357
Epoch 66
Validation auc = 0.652113
Epoch 67
Validation auc = 0.651174
Epoch 68
Validation auc = 0.649296
Epoch 69
Loss = 5.0683e-01, PNorm = 41.2254, GNorm = 2.8638, lr_0 = 8.9978e-04
Validation auc = 0.648357
Epoch 70
Validation auc = 0.641784
Epoch 71
Validation auc = 0.638498
Epoch 72
Validation auc = 0.629577
Epoch 73
Validation auc = 0.630516
Epoch 74
Loss = 5.5573e-01, PNorm = 41.2940, GNorm = 3.4333, lr_0 = 8.9209e-04
Validation auc = 0.637089
Epoch 75
Validation auc = 0.648357
Epoch 76
Validation auc = 0.658685
Epoch 77
Validation auc = 0.659624
Epoch 78
Validation auc = 0.653991
Epoch 79
Loss = 5.9418e-01, PNorm = 41.3519, GNorm = 2.8465, lr_0 = 8.8447e-04
Validation auc = 0.635211
Epoch 80
Validation auc = 0.623474
Epoch 81
Validation auc = 0.617840
Epoch 82
Validation auc = 0.624883
Epoch 83
Validation auc = 0.633803
Epoch 84
Loss = 4.4913e-01, PNorm = 41.4261, GNorm = 2.9812, lr_0 = 8.7691e-04
Validation auc = 0.644131
Epoch 85
Validation auc = 0.644601
Epoch 86
Validation auc = 0.630986
Epoch 87
Validation auc = 0.631455
Epoch 88
Validation auc = 0.634272
Epoch 89
Loss = 5.2753e-01, PNorm = 41.4811, GNorm = 3.4263, lr_0 = 8.6941e-04
Validation auc = 0.638028
Epoch 90
Validation auc = 0.640376
Epoch 91
Validation auc = 0.637089
Epoch 92
Validation auc = 0.642723
Epoch 93
Validation auc = 0.645070
Epoch 94
Loss = 4.1392e-01, PNorm = 41.5556, GNorm = 2.8698, lr_0 = 8.6198e-04
Validation auc = 0.648826
Epoch 95
Validation auc = 0.652582
Epoch 96
Validation auc = 0.653521
Epoch 97
Validation auc = 0.659155
Epoch 98
Validation auc = 0.654930
Epoch 99
Loss = 4.4405e-01, PNorm = 41.6218, GNorm = 3.5428, lr_0 = 8.5461e-04
Validation auc = 0.658216
Epoch 100
Validation auc = 0.647887
Epoch 101
Validation auc = 0.644601
Epoch 102
Validation auc = 0.646948
Epoch 103
Validation auc = 0.650704
Epoch 104
Loss = 5.5670e-01, PNorm = 41.6764, GNorm = 3.3436, lr_0 = 8.4730e-04
Validation auc = 0.645540
Epoch 105
Validation auc = 0.647887
Epoch 106
Validation auc = 0.657277
Epoch 107
Validation auc = 0.664319
Epoch 108
Validation auc = 0.675587
Epoch 109
Loss = 4.8457e-01, PNorm = 41.7572, GNorm = 3.0200, lr_0 = 8.4006e-04
Validation auc = 0.669014
Epoch 110
Validation auc = 0.661033
Epoch 111
Validation auc = 0.668075
Epoch 112
Validation auc = 0.671362
Epoch 113
Validation auc = 0.676056
Epoch 114
Loss = 4.9363e-01, PNorm = 41.8322, GNorm = 2.5894, lr_0 = 8.3288e-04
Validation auc = 0.684038
Epoch 115
Validation auc = 0.688732
Epoch 116
Validation auc = 0.680282
Epoch 117
Validation auc = 0.673709
Epoch 118
Validation auc = 0.661033
Epoch 119
Loss = 5.2572e-01, PNorm = 41.9128, GNorm = 2.5399, lr_0 = 8.2576e-04
Validation auc = 0.661502
Epoch 120
Validation auc = 0.662441
Epoch 121
Validation auc = 0.666197
Epoch 122
Validation auc = 0.669484
Epoch 123
Validation auc = 0.660094
Epoch 124
Loss = 4.9607e-01, PNorm = 42.0133, GNorm = 2.9986, lr_0 = 8.1870e-04
Validation auc = 0.657746
Epoch 125
Validation auc = 0.661972
Epoch 126
Validation auc = 0.678404
Epoch 127
Validation auc = 0.670423
Epoch 128
Validation auc = 0.661033
Epoch 129
Loss = 3.9704e-01, PNorm = 42.1044, GNorm = 2.5767, lr_0 = 8.1170e-04
Validation auc = 0.659155
Epoch 130
Validation auc = 0.662911
Epoch 131
Validation auc = 0.667136
Epoch 132
Validation auc = 0.672300
Epoch 133
Validation auc = 0.683099
Epoch 134
Loss = 4.0641e-01, PNorm = 42.2112, GNorm = 2.7935, lr_0 = 8.0476e-04
Validation auc = 0.700000
Epoch 135
Validation auc = 0.689671
Epoch 136
Validation auc = 0.687324
Epoch 137
Validation auc = 0.682629
Epoch 138
Validation auc = 0.678873
Epoch 139
Loss = 4.5826e-01, PNorm = 42.2806, GNorm = 4.3042, lr_0 = 7.9788e-04
Validation auc = 0.684977
Epoch 140
Validation auc = 0.693427
Epoch 141
Validation auc = 0.693897
Epoch 142
Validation auc = 0.694366
Epoch 143
Validation auc = 0.696244
Epoch 144
Loss = 4.8512e-01, PNorm = 42.3697, GNorm = 3.2741, lr_0 = 7.9106e-04
Validation auc = 0.704695
Epoch 145
Validation auc = 0.705634
Epoch 146
Validation auc = 0.713615
Epoch 147
Validation auc = 0.714554
Epoch 148
Validation auc = 0.716432
Epoch 149
Loss = 4.7121e-01, PNorm = 42.4670, GNorm = 4.2043, lr_0 = 7.8430e-04
Validation auc = 0.710798
Epoch 150
Validation auc = 0.705634
Epoch 151
Validation auc = 0.695775
Epoch 152
Validation auc = 0.694366
Epoch 153
Validation auc = 0.704695
Epoch 154
Loss = 3.6228e-01, PNorm = 42.5743, GNorm = 2.4236, lr_0 = 7.7759e-04
Validation auc = 0.704695
Epoch 155
Validation auc = 0.715493
Epoch 156
Validation auc = 0.724413
Epoch 157
Validation auc = 0.724883
Epoch 158
Validation auc = 0.724883
Epoch 159
Loss = 5.3620e-01, PNorm = 42.6870, GNorm = 4.2912, lr_0 = 7.7095e-04
Validation auc = 0.724413
Epoch 160
Validation auc = 0.727700
Epoch 161
Validation auc = 0.720188
Epoch 162
Validation auc = 0.713146
Epoch 163
Validation auc = 0.709390
Epoch 164
Loss = 3.9207e-01, PNorm = 42.7686, GNorm = 3.4823, lr_0 = 7.6436e-04
Validation auc = 0.710329
Epoch 165
Validation auc = 0.708451
Epoch 166
Validation auc = 0.709390
Epoch 167
Validation auc = 0.702347
Epoch 168
Validation auc = 0.702817
Epoch 169
Loss = 3.3011e-01, PNorm = 42.8708, GNorm = 2.2544, lr_0 = 7.5782e-04
Validation auc = 0.705164
Epoch 170
Validation auc = 0.704695
Epoch 171
Validation auc = 0.702817
Epoch 172
Validation auc = 0.707512
Epoch 173
Validation auc = 0.712676
Epoch 174
Loss = 3.4080e-01, PNorm = 42.9855, GNorm = 2.7074, lr_0 = 7.5134e-04
Validation auc = 0.709390
Epoch 175
Validation auc = 0.710329
Epoch 176
Validation auc = 0.713146
Epoch 177
Validation auc = 0.714554
Epoch 178
Validation auc = 0.715493
Epoch 179
Loss = 3.6012e-01, PNorm = 43.0959, GNorm = 2.5332, lr_0 = 7.4492e-04
Validation auc = 0.713615
Epoch 180
Validation auc = 0.719249
Epoch 181
Validation auc = 0.723474
Epoch 182
Validation auc = 0.715962
Epoch 183
Validation auc = 0.715023
Epoch 184
Loss = 4.3139e-01, PNorm = 43.1846, GNorm = 3.0004, lr_0 = 7.3855e-04
Validation auc = 0.712207
Epoch 185
Validation auc = 0.708920
Epoch 186
Validation auc = 0.715493
Epoch 187
Validation auc = 0.718310
Epoch 188
Validation auc = 0.721596
Epoch 189
Loss = 4.0971e-01, PNorm = 43.2773, GNorm = 4.3169, lr_0 = 7.3224e-04
Validation auc = 0.722066
Epoch 190
Validation auc = 0.707981
Epoch 191
Validation auc = 0.704695
Epoch 192
Validation auc = 0.713146
Epoch 193
Validation auc = 0.715493
Epoch 194
Loss = 4.0989e-01, PNorm = 43.3738, GNorm = 3.6659, lr_0 = 7.2598e-04
Validation auc = 0.717371
Epoch 195
Validation auc = 0.721127
Epoch 196
Validation auc = 0.722535
Epoch 197
Validation auc = 0.723944
Epoch 198
Validation auc = 0.718779
Epoch 199
Loss = 3.4719e-01, PNorm = 43.4653, GNorm = 2.5299, lr_0 = 7.1977e-04
Validation auc = 0.720657
Epoch 200
Validation auc = 0.715023
Epoch 201
Validation auc = 0.710329
Epoch 202
Validation auc = 0.712676
Epoch 203
Validation auc = 0.712207
Epoch 204
Loss = 3.1557e-01, PNorm = 43.5613, GNorm = 3.0541, lr_0 = 7.1362e-04
Validation auc = 0.713146
Epoch 205
Validation auc = 0.715493
Epoch 206
Validation auc = 0.715962
Epoch 207
Validation auc = 0.719249
Epoch 208
Validation auc = 0.719249
Epoch 209
Loss = 3.7296e-01, PNorm = 43.6522, GNorm = 3.5219, lr_0 = 7.0752e-04
Validation auc = 0.717840
Epoch 210
Validation auc = 0.715962
Epoch 211
Validation auc = 0.720657
Epoch 212
Validation auc = 0.719718
Epoch 213
Validation auc = 0.720188
Epoch 214
Loss = 3.6599e-01, PNorm = 43.7607, GNorm = 2.3538, lr_0 = 7.0147e-04
Validation auc = 0.723474
Epoch 215
Validation auc = 0.727230
Epoch 216
Validation auc = 0.728169
Epoch 217
Validation auc = 0.725352
Epoch 218
Validation auc = 0.726761
Epoch 219
Loss = 3.6650e-01, PNorm = 43.8413, GNorm = 2.9062, lr_0 = 6.9548e-04
Validation auc = 0.729108
Epoch 220
Validation auc = 0.730047
Epoch 221
Validation auc = 0.732394
Epoch 222
Validation auc = 0.735211
Epoch 223
Validation auc = 0.727700
Epoch 224
Loss = 3.4001e-01, PNorm = 43.9360, GNorm = 3.8500, lr_0 = 6.8953e-04
Validation auc = 0.722066
Epoch 225
Validation auc = 0.712207
Epoch 226
Validation auc = 0.714554
Epoch 227
Validation auc = 0.716901
Epoch 228
Validation auc = 0.724883
Epoch 229
Loss = 3.3092e-01, PNorm = 44.0463, GNorm = 2.6432, lr_0 = 6.8364e-04
Validation auc = 0.723944
Epoch 230
Validation auc = 0.726291
Epoch 231
Validation auc = 0.726761
Epoch 232
Validation auc = 0.729108
Epoch 233
Validation auc = 0.731455
Epoch 234
Loss = 2.2783e-01, PNorm = 44.1461, GNorm = 1.5628, lr_0 = 6.7779e-04
Validation auc = 0.730986
Epoch 235
Validation auc = 0.729577
Epoch 236
Validation auc = 0.729108
Epoch 237
Validation auc = 0.733333
Epoch 238
Validation auc = 0.735681
Epoch 239
Loss = 3.5763e-01, PNorm = 44.2374, GNorm = 2.4709, lr_0 = 6.7200e-04
Validation auc = 0.747418
Epoch 240
Validation auc = 0.762911
Epoch 241
Validation auc = 0.755399
Epoch 242
Validation auc = 0.747887
Epoch 243
Validation auc = 0.745070
Epoch 244
Loss = 3.6791e-01, PNorm = 44.3307, GNorm = 3.2160, lr_0 = 6.6625e-04
Validation auc = 0.734272
Epoch 245
Validation auc = 0.734742
Epoch 246
Validation auc = 0.729577
Epoch 247
Validation auc = 0.730047
Epoch 248
Validation auc = 0.727230
Epoch 249
Loss = 2.8694e-01, PNorm = 44.4332, GNorm = 1.6074, lr_0 = 6.6056e-04
Validation auc = 0.728638
Epoch 250
Validation auc = 0.727230
Epoch 251
Validation auc = 0.724883
Epoch 252
Validation auc = 0.723944
Epoch 253
Validation auc = 0.721596
Epoch 254
Loss = 2.2079e-01, PNorm = 44.5318, GNorm = 1.7543, lr_0 = 6.5491e-04
Validation auc = 0.722535
Epoch 255
Validation auc = 0.724883
Epoch 256
Validation auc = 0.727230
Epoch 257
Validation auc = 0.728638
Epoch 258
Validation auc = 0.724413
Epoch 259
Loss = 2.3631e-01, PNorm = 44.6401, GNorm = 3.1860, lr_0 = 6.4931e-04
Validation auc = 0.721596
Epoch 260
Validation auc = 0.723005
Epoch 261
Validation auc = 0.723005
Epoch 262
Validation auc = 0.722066
Epoch 263
Validation auc = 0.721596
Epoch 264
Loss = 2.1751e-01, PNorm = 44.7279, GNorm = 2.6049, lr_0 = 6.4376e-04
Validation auc = 0.721127
Epoch 265
Validation auc = 0.723005
Epoch 266
Validation auc = 0.724413
Epoch 267
Validation auc = 0.720657
Epoch 268
Validation auc = 0.720188
Epoch 269
Loss = 2.7057e-01, PNorm = 44.8017, GNorm = 2.4837, lr_0 = 6.3826e-04
Validation auc = 0.715023
Epoch 270
Validation auc = 0.709390
Epoch 271
Validation auc = 0.710329
Epoch 272
Validation auc = 0.707981
Epoch 273
Validation auc = 0.710329
Epoch 274
Loss = 2.6608e-01, PNorm = 44.8758, GNorm = 3.4665, lr_0 = 6.3280e-04
Validation auc = 0.713146
Epoch 275
Validation auc = 0.716901
Epoch 276
Validation auc = 0.725352
Epoch 277
Validation auc = 0.729108
Epoch 278
Validation auc = 0.737559
Epoch 279
Loss = 2.7600e-01, PNorm = 44.9589, GNorm = 2.8883, lr_0 = 6.2739e-04
Validation auc = 0.739906
Epoch 280
Validation auc = 0.737559
Epoch 281
Validation auc = 0.735211
Epoch 282
Validation auc = 0.723944
Epoch 283
Validation auc = 0.722066
Epoch 284
Loss = 2.5160e-01, PNorm = 45.0346, GNorm = 1.8325, lr_0 = 6.2203e-04
Validation auc = 0.726761
Epoch 285
Validation auc = 0.728169
Epoch 286
Validation auc = 0.723944
Epoch 287
Validation auc = 0.723005
Epoch 288
Validation auc = 0.728169
Epoch 289
Loss = 2.3684e-01, PNorm = 45.1250, GNorm = 3.1984, lr_0 = 6.1671e-04
Validation auc = 0.733803
Epoch 290
Validation auc = 0.739906
Epoch 291
Validation auc = 0.745070
Epoch 292
Validation auc = 0.749765
Epoch 293
Validation auc = 0.751643
Epoch 294
Loss = 2.8611e-01, PNorm = 45.2016, GNorm = 2.7867, lr_0 = 6.1144e-04
Validation auc = 0.745540
Epoch 295
Validation auc = 0.743192
Epoch 296
Validation auc = 0.741315
Epoch 297
Validation auc = 0.743662
Epoch 298
Validation auc = 0.737559
Epoch 299
Loss = 2.0096e-01, PNorm = 45.2722, GNorm = 1.6293, lr_0 = 6.0621e-04
Validation auc = 0.729577
Model 0 best validation auc = 0.762911 on epoch 240
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.921734
Ensemble test auc = 0.921734
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.507981
Epoch 1
Validation auc = 0.681221
Epoch 2
Validation auc = 0.676056
Epoch 3
Validation auc = 0.688263
Epoch 4
Loss = 9.2551e-01, PNorm = 40.6666, GNorm = 9.8156, lr_0 = 6.5000e-04
Validation auc = 0.712676
Epoch 5
Validation auc = 0.705634
Epoch 6
Validation auc = 0.690141
Epoch 7
Validation auc = 0.677934
Epoch 8
Validation auc = 0.653521
Epoch 9
Loss = 8.5648e-01, PNorm = 40.7162, GNorm = 9.7049, lr_0 = 9.9743e-04
Validation auc = 0.631455
Epoch 10
Validation auc = 0.638028
Epoch 11
Validation auc = 0.617840
Epoch 12
Validation auc = 0.646009
Epoch 13
Validation auc = 0.629577
Epoch 14
Loss = 9.4990e-01, PNorm = 40.7607, GNorm = 7.5925, lr_0 = 9.8890e-04
Validation auc = 0.665258
Epoch 15
Validation auc = 0.697653
Epoch 16
Validation auc = 0.716901
Epoch 17
Validation auc = 0.678404
Epoch 18
Validation auc = 0.682160
Epoch 19
Loss = 7.0413e-01, PNorm = 40.7954, GNorm = 5.6336, lr_0 = 9.8045e-04
Validation auc = 0.703756
Epoch 20
Validation auc = 0.786854
Epoch 21
Validation auc = 0.835681
Epoch 22
Validation auc = 0.815023
Epoch 23
Validation auc = 0.711268
Epoch 24
Loss = 6.5052e-01, PNorm = 40.8276, GNorm = 5.5954, lr_0 = 9.7207e-04
Validation auc = 0.698122
Epoch 25
Validation auc = 0.700939
Epoch 26
Validation auc = 0.710798
Epoch 27
Validation auc = 0.694366
Epoch 28
Validation auc = 0.687793
Epoch 29
Loss = 5.5701e-01, PNorm = 40.8621, GNorm = 5.6477, lr_0 = 9.6376e-04
Validation auc = 0.676056
Epoch 30
Validation auc = 0.666197
Epoch 31
Validation auc = 0.667606
Epoch 32
Validation auc = 0.688732
Epoch 33
Validation auc = 0.731925
Epoch 34
Loss = 6.4863e-01, PNorm = 40.8961, GNorm = 5.7479, lr_0 = 9.5552e-04
Validation auc = 0.715493
Epoch 35
Validation auc = 0.692958
Epoch 36
Validation auc = 0.692488
Epoch 37
Validation auc = 0.688732
Epoch 38
Validation auc = 0.714554
Epoch 39
Loss = 5.7298e-01, PNorm = 40.9312, GNorm = 4.5262, lr_0 = 9.4735e-04
Validation auc = 0.747418
Epoch 40
Validation auc = 0.759155
Epoch 41
Validation auc = 0.740376
Epoch 42
Validation auc = 0.721127
Epoch 43
Validation auc = 0.726291
Epoch 44
Loss = 5.9993e-01, PNorm = 40.9665, GNorm = 5.0135, lr_0 = 9.3925e-04
Validation auc = 0.739437
Epoch 45
Validation auc = 0.732394
Epoch 46
Validation auc = 0.705634
Epoch 47
Validation auc = 0.694836
Epoch 48
Validation auc = 0.690141
Epoch 49
Loss = 5.9951e-01, PNorm = 41.0116, GNorm = 4.7488, lr_0 = 9.3122e-04
Validation auc = 0.686385
Epoch 50
Validation auc = 0.687324
Epoch 51
Validation auc = 0.692488
Epoch 52
Validation auc = 0.690610
Epoch 53
Validation auc = 0.684977
Epoch 54
Loss = 5.1889e-01, PNorm = 41.0583, GNorm = 3.1189, lr_0 = 9.2326e-04
Validation auc = 0.687324
Epoch 55
Validation auc = 0.689202
Epoch 56
Validation auc = 0.685915
Epoch 57
Validation auc = 0.681690
Epoch 58
Validation auc = 0.683568
Epoch 59
Loss = 6.2089e-01, PNorm = 41.1031, GNorm = 4.3007, lr_0 = 9.1537e-04
Validation auc = 0.656808
Epoch 60
Validation auc = 0.625822
Epoch 61
Validation auc = 0.605634
Epoch 62
Validation auc = 0.601408
Epoch 63
Validation auc = 0.603286
Epoch 64
Loss = 5.4972e-01, PNorm = 41.1586, GNorm = 4.7985, lr_0 = 9.0754e-04
Validation auc = 0.607042
Epoch 65
Validation auc = 0.615493
Epoch 66
Validation auc = 0.622535
Epoch 67
Validation auc = 0.623005
Epoch 68
Validation auc = 0.616432
Epoch 69
Loss = 5.5597e-01, PNorm = 41.2086, GNorm = 2.8667, lr_0 = 8.9978e-04
Validation auc = 0.607981
Epoch 70
Validation auc = 0.601408
Epoch 71
Validation auc = 0.599531
Epoch 72
Validation auc = 0.602817
Epoch 73
Validation auc = 0.598122
Epoch 74
Loss = 5.6000e-01, PNorm = 41.2612, GNorm = 3.0247, lr_0 = 8.9209e-04
Validation auc = 0.609390
Epoch 75
Validation auc = 0.622535
Epoch 76
Validation auc = 0.626761
Epoch 77
Validation auc = 0.623474
Epoch 78
Validation auc = 0.613146
Epoch 79
Loss = 4.7712e-01, PNorm = 41.3117, GNorm = 3.6181, lr_0 = 8.8447e-04
Validation auc = 0.623005
Epoch 80
Validation auc = 0.630047
Epoch 81
Validation auc = 0.631455
Epoch 82
Validation auc = 0.638498
Epoch 83
Validation auc = 0.645540
Epoch 84
Loss = 5.3522e-01, PNorm = 41.3727, GNorm = 2.7674, lr_0 = 8.7691e-04
Validation auc = 0.649765
Epoch 85
Validation auc = 0.642254
Epoch 86
Validation auc = 0.646009
Epoch 87
Validation auc = 0.657746
Epoch 88
Validation auc = 0.670423
Epoch 89
Loss = 5.7134e-01, PNorm = 41.4322, GNorm = 3.1952, lr_0 = 8.6941e-04
Validation auc = 0.676526
Epoch 90
Validation auc = 0.684038
Epoch 91
Validation auc = 0.690610
Epoch 92
Validation auc = 0.699061
Epoch 93
Validation auc = 0.698592
Epoch 94
Loss = 4.8302e-01, PNorm = 41.5126, GNorm = 3.6096, lr_0 = 8.6198e-04
Validation auc = 0.701408
Epoch 95
Validation auc = 0.696714
Epoch 96
Validation auc = 0.689671
Epoch 97
Validation auc = 0.686385
Epoch 98
Validation auc = 0.681690
Epoch 99
Loss = 4.8676e-01, PNorm = 41.5784, GNorm = 3.5674, lr_0 = 8.5461e-04
Validation auc = 0.688732
Epoch 100
Validation auc = 0.700000
Epoch 101
Validation auc = 0.707981
Epoch 102
Validation auc = 0.700939
Epoch 103
Validation auc = 0.700000
Epoch 104
Loss = 5.0546e-01, PNorm = 41.6618, GNorm = 3.1421, lr_0 = 8.4730e-04
Validation auc = 0.696714
Epoch 105
Validation auc = 0.693897
Epoch 106
Validation auc = 0.690141
Epoch 107
Validation auc = 0.673709
Epoch 108
Validation auc = 0.663850
Epoch 109
Loss = 4.4904e-01, PNorm = 41.7526, GNorm = 3.4140, lr_0 = 8.4006e-04
Validation auc = 0.666667
Epoch 110
Validation auc = 0.659155
Epoch 111
Validation auc = 0.646479
Epoch 112
Validation auc = 0.663850
Epoch 113
Validation auc = 0.661972
Epoch 114
Loss = 6.0960e-01, PNorm = 41.8488, GNorm = 4.5595, lr_0 = 8.3288e-04
Validation auc = 0.660094
Epoch 115
Validation auc = 0.659155
Epoch 116
Validation auc = 0.658216
Epoch 117
Validation auc = 0.683099
Epoch 118
Validation auc = 0.678873
Epoch 119
Loss = 4.0929e-01, PNorm = 41.9473, GNorm = 2.5357, lr_0 = 8.2576e-04
Validation auc = 0.666667
Epoch 120
Validation auc = 0.644601
Epoch 121
Validation auc = 0.642723
Epoch 122
Validation auc = 0.639437
Epoch 123
Validation auc = 0.646479
Epoch 124
Loss = 5.2056e-01, PNorm = 42.0355, GNorm = 3.8653, lr_0 = 8.1870e-04
Validation auc = 0.666197
Epoch 125
Validation auc = 0.681221
Epoch 126
Validation auc = 0.693897
Epoch 127
Validation auc = 0.696714
Epoch 128
Validation auc = 0.694836
Epoch 129
Loss = 4.1412e-01, PNorm = 42.1342, GNorm = 1.8463, lr_0 = 8.1170e-04
Validation auc = 0.692019
Epoch 130
Validation auc = 0.685915
Epoch 131
Validation auc = 0.703756
Epoch 132
Validation auc = 0.708920
Epoch 133
Validation auc = 0.719718
Epoch 134
Loss = 4.9161e-01, PNorm = 42.2235, GNorm = 3.0690, lr_0 = 8.0476e-04
Validation auc = 0.724413
Epoch 135
Validation auc = 0.727700
Epoch 136
Validation auc = 0.729108
Epoch 137
Validation auc = 0.722535
Epoch 138
Validation auc = 0.710329
Epoch 139
Loss = 4.2085e-01, PNorm = 42.3326, GNorm = 4.0519, lr_0 = 7.9788e-04
Validation auc = 0.712207
Epoch 140
Validation auc = 0.708451
Epoch 141
Validation auc = 0.703286
Epoch 142
Validation auc = 0.721596
Epoch 143
Validation auc = 0.714554
Epoch 144
Loss = 4.4986e-01, PNorm = 42.4493, GNorm = 2.5090, lr_0 = 7.9106e-04
Validation auc = 0.726291
Epoch 145
Validation auc = 0.722535
Epoch 146
Validation auc = 0.721596
Epoch 147
Validation auc = 0.726761
Epoch 148
Validation auc = 0.735681
Epoch 149
Loss = 4.9168e-01, PNorm = 42.5552, GNorm = 3.1605, lr_0 = 7.8430e-04
Validation auc = 0.733803
Epoch 150
Validation auc = 0.739906
Epoch 151
Validation auc = 0.741784
Epoch 152
Validation auc = 0.735211
Epoch 153
Validation auc = 0.730516
Epoch 154
Loss = 3.2871e-01, PNorm = 42.6718, GNorm = 3.0211, lr_0 = 7.7759e-04
Validation auc = 0.724413
Epoch 155
Validation auc = 0.725352
Epoch 156
Validation auc = 0.732394
Epoch 157
Validation auc = 0.731455
Epoch 158
Validation auc = 0.750704
Epoch 159
Loss = 3.9675e-01, PNorm = 42.7862, GNorm = 4.1472, lr_0 = 7.7095e-04
Validation auc = 0.751643
Epoch 160
Validation auc = 0.734742
Epoch 161
Validation auc = 0.731925
Epoch 162
Validation auc = 0.730516
Epoch 163
Validation auc = 0.727230
Epoch 164
Loss = 4.0452e-01, PNorm = 42.8710, GNorm = 4.3885, lr_0 = 7.6436e-04
Validation auc = 0.727700
Epoch 165
Validation auc = 0.730047
Epoch 166
Validation auc = 0.739437
Epoch 167
Validation auc = 0.747887
Epoch 168
Validation auc = 0.748357
Epoch 169
Loss = 3.0051e-01, PNorm = 42.9763, GNorm = 2.3212, lr_0 = 7.5782e-04
Validation auc = 0.740376
Epoch 170
Validation auc = 0.731455
Epoch 171
Validation auc = 0.723944
Epoch 172
Validation auc = 0.713146
Epoch 173
Validation auc = 0.707512
Epoch 174
Loss = 3.6762e-01, PNorm = 43.0772, GNorm = 3.2309, lr_0 = 7.5134e-04
Validation auc = 0.711268
Epoch 175
Validation auc = 0.719249
Epoch 176
Validation auc = 0.721127
Epoch 177
Validation auc = 0.733333
Epoch 178
Validation auc = 0.739437
Epoch 179
Loss = 3.5989e-01, PNorm = 43.1597, GNorm = 2.8168, lr_0 = 7.4492e-04
Validation auc = 0.739906
Epoch 180
Validation auc = 0.743662
Epoch 181
Validation auc = 0.745070
Epoch 182
Validation auc = 0.746948
Epoch 183
Validation auc = 0.733803
Epoch 184
Loss = 4.0502e-01, PNorm = 43.2428, GNorm = 2.9526, lr_0 = 7.3855e-04
Validation auc = 0.723944
Epoch 185
Validation auc = 0.721127
Epoch 186
Validation auc = 0.718779
Epoch 187
Validation auc = 0.722066
Epoch 188
Validation auc = 0.733333
Epoch 189
Loss = 2.6426e-01, PNorm = 43.3387, GNorm = 1.7376, lr_0 = 7.3224e-04
Validation auc = 0.746009
Epoch 190
Validation auc = 0.753521
Epoch 191
Validation auc = 0.756808
Epoch 192
Validation auc = 0.765258
Epoch 193
Validation auc = 0.761033
Epoch 194
Loss = 2.9796e-01, PNorm = 43.4258, GNorm = 2.2773, lr_0 = 7.2598e-04
Validation auc = 0.762441
Epoch 195
Validation auc = 0.758216
Epoch 196
Validation auc = 0.750704
Epoch 197
Validation auc = 0.745070
Epoch 198
Validation auc = 0.740376
Epoch 199
Loss = 3.3403e-01, PNorm = 43.5011, GNorm = 1.9868, lr_0 = 7.1977e-04
Validation auc = 0.744131
Epoch 200
Validation auc = 0.746948
Epoch 201
Validation auc = 0.750704
Epoch 202
Validation auc = 0.759155
Epoch 203
Validation auc = 0.765258
Epoch 204
Loss = 2.7899e-01, PNorm = 43.5770, GNorm = 2.0935, lr_0 = 7.1362e-04
Validation auc = 0.769484
Epoch 205
Validation auc = 0.761502
Epoch 206
Validation auc = 0.749765
Epoch 207
Validation auc = 0.749296
Epoch 208
Validation auc = 0.748826
Epoch 209
Loss = 3.3478e-01, PNorm = 43.6682, GNorm = 2.7318, lr_0 = 7.0752e-04
Validation auc = 0.747418
Epoch 210
Validation auc = 0.747887
Epoch 211
Validation auc = 0.749296
Epoch 212
Validation auc = 0.753521
Epoch 213
Validation auc = 0.757277
Epoch 214
Loss = 3.6373e-01, PNorm = 43.7502, GNorm = 3.2662, lr_0 = 7.0147e-04
Validation auc = 0.754930
Epoch 215
Validation auc = 0.748826
Epoch 216
Validation auc = 0.739437
Epoch 217
Validation auc = 0.734272
Epoch 218
Validation auc = 0.733333
Epoch 219
Loss = 2.8889e-01, PNorm = 43.8212, GNorm = 1.4596, lr_0 = 6.9548e-04
Validation auc = 0.729577
Epoch 220
Validation auc = 0.721596
Epoch 221
Validation auc = 0.717840
Epoch 222
Validation auc = 0.715962
Epoch 223
Validation auc = 0.714554
Epoch 224
Loss = 2.7272e-01, PNorm = 43.8994, GNorm = 3.6426, lr_0 = 6.8953e-04
Validation auc = 0.718310
Epoch 225
Validation auc = 0.723005
Epoch 226
Validation auc = 0.727700
Epoch 227
Validation auc = 0.737559
Epoch 228
Validation auc = 0.740376
Epoch 229
Loss = 3.6931e-01, PNorm = 43.9625, GNorm = 2.3944, lr_0 = 6.8364e-04
Validation auc = 0.741315
Epoch 230
Validation auc = 0.744131
Epoch 231
Validation auc = 0.753521
Epoch 232
Validation auc = 0.766667
Epoch 233
Validation auc = 0.768545
Epoch 234
Loss = 4.3187e-01, PNorm = 44.0352, GNorm = 2.0812, lr_0 = 6.7779e-04
Validation auc = 0.762441
Epoch 235
Validation auc = 0.765258
Epoch 236
Validation auc = 0.766197
Epoch 237
Validation auc = 0.767136
Epoch 238
Validation auc = 0.759155
Epoch 239
Loss = 3.2616e-01, PNorm = 44.1099, GNorm = 3.8610, lr_0 = 6.7200e-04
Validation auc = 0.758685
Epoch 240
Validation auc = 0.753521
Epoch 241
Validation auc = 0.759624
Epoch 242
Validation auc = 0.758685
Epoch 243
Validation auc = 0.751174
Epoch 244
Loss = 2.8388e-01, PNorm = 44.1916, GNorm = 1.6452, lr_0 = 6.6625e-04
Validation auc = 0.747887
Epoch 245
Validation auc = 0.735211
Epoch 246
Validation auc = 0.732864
Epoch 247
Validation auc = 0.732864
Epoch 248
Validation auc = 0.737089
Epoch 249
Loss = 3.0000e-01, PNorm = 44.2691, GNorm = 2.0629, lr_0 = 6.6056e-04
Validation auc = 0.737559
Epoch 250
Validation auc = 0.740376
Epoch 251
Validation auc = 0.745070
Epoch 252
Validation auc = 0.740845
Epoch 253
Validation auc = 0.740376
Epoch 254
Loss = 3.1042e-01, PNorm = 44.3411, GNorm = 3.2057, lr_0 = 6.5491e-04
Validation auc = 0.740845
Epoch 255
Validation auc = 0.742254
Epoch 256
Validation auc = 0.739437
Epoch 257
Validation auc = 0.736620
Epoch 258
Validation auc = 0.736620
Epoch 259
Loss = 2.8796e-01, PNorm = 44.4145, GNorm = 2.3813, lr_0 = 6.4931e-04
Validation auc = 0.732864
Epoch 260
Validation auc = 0.738498
Epoch 261
Validation auc = 0.748826
Epoch 262
Validation auc = 0.745540
Epoch 263
Validation auc = 0.738028
Epoch 264
Loss = 2.8911e-01, PNorm = 44.4792, GNorm = 2.8896, lr_0 = 6.4376e-04
Validation auc = 0.734742
Epoch 265
Validation auc = 0.731925
Epoch 266
Validation auc = 0.735211
Epoch 267
Validation auc = 0.735211
Epoch 268
Validation auc = 0.735681
Epoch 269
Loss = 2.7430e-01, PNorm = 44.5443, GNorm = 1.3776, lr_0 = 6.3826e-04
Validation auc = 0.737559
Epoch 270
Validation auc = 0.739906
Epoch 271
Validation auc = 0.740845
Epoch 272
Validation auc = 0.743192
Epoch 273
Validation auc = 0.744601
Epoch 274
Loss = 2.1225e-01, PNorm = 44.6094, GNorm = 1.6614, lr_0 = 6.3280e-04
Validation auc = 0.739437
Epoch 275
Validation auc = 0.738028
Epoch 276
Validation auc = 0.741315
Epoch 277
Validation auc = 0.744601
Epoch 278
Validation auc = 0.745540
Epoch 279
Loss = 2.6938e-01, PNorm = 44.6797, GNorm = 1.4283, lr_0 = 6.2739e-04
Validation auc = 0.737559
Epoch 280
Validation auc = 0.730516
Epoch 281
Validation auc = 0.730516
Epoch 282
Validation auc = 0.737089
Epoch 283
Validation auc = 0.753521
Epoch 284
Loss = 3.3574e-01, PNorm = 44.7613, GNorm = 3.8172, lr_0 = 6.2203e-04
Validation auc = 0.755869
Epoch 285
Validation auc = 0.750235
Epoch 286
Validation auc = 0.746948
Epoch 287
Validation auc = 0.748357
Epoch 288
Validation auc = 0.751174
Epoch 289
Loss = 2.5106e-01, PNorm = 44.8312, GNorm = 1.5080, lr_0 = 6.1671e-04
Validation auc = 0.758216
Epoch 290
Validation auc = 0.756338
Epoch 291
Validation auc = 0.758216
Epoch 292
Validation auc = 0.755399
Epoch 293
Validation auc = 0.751174
Epoch 294
Loss = 2.6446e-01, PNorm = 44.9100, GNorm = 2.0535, lr_0 = 6.1144e-04
Validation auc = 0.753052
Epoch 295
Validation auc = 0.746948
Epoch 296
Validation auc = 0.746479
Epoch 297
Validation auc = 0.744131
Epoch 298
Validation auc = 0.736620
Epoch 299
Loss = 2.6127e-01, PNorm = 44.9815, GNorm = 4.4577, lr_0 = 6.0621e-04
Validation auc = 0.735681
Model 0 best validation auc = 0.835681 on epoch 21
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.797110
Ensemble test auc = 0.797110
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.511268
Epoch 1
Validation auc = 0.688263
Epoch 2
Validation auc = 0.684038
Epoch 3
Validation auc = 0.703286
Epoch 4
Loss = 1.0335e+00, PNorm = 40.6655, GNorm = 13.8268, lr_0 = 6.5000e-04
Validation auc = 0.715962
Epoch 5
Validation auc = 0.721596
Epoch 6
Validation auc = 0.666667
Epoch 7
Validation auc = 0.661972
Epoch 8
Validation auc = 0.644601
Epoch 9
Loss = 1.0849e+00, PNorm = 40.7127, GNorm = 12.4416, lr_0 = 9.9743e-04
Validation auc = 0.644601
Epoch 10
Validation auc = 0.657746
Epoch 11
Validation auc = 0.652582
Epoch 12
Validation auc = 0.650704
Epoch 13
Validation auc = 0.638967
Epoch 14
Loss = 7.3438e-01, PNorm = 40.7623, GNorm = 6.9317, lr_0 = 9.8890e-04
Validation auc = 0.639437
Epoch 15
Validation auc = 0.650704
Epoch 16
Validation auc = 0.652113
Epoch 17
Validation auc = 0.650704
Epoch 18
Validation auc = 0.688263
Epoch 19
Loss = 7.3504e-01, PNorm = 40.8110, GNorm = 5.9748, lr_0 = 9.8045e-04
Validation auc = 0.710798
Epoch 20
Validation auc = 0.697183
Epoch 21
Validation auc = 0.694366
Epoch 22
Validation auc = 0.687324
Epoch 23
Validation auc = 0.689202
Epoch 24
Loss = 7.7267e-01, PNorm = 40.8503, GNorm = 8.7644, lr_0 = 9.7207e-04
Validation auc = 0.704225
Epoch 25
Validation auc = 0.701878
Epoch 26
Validation auc = 0.707042
Epoch 27
Validation auc = 0.713615
Epoch 28
Validation auc = 0.696714
Epoch 29
Loss = 6.7391e-01, PNorm = 40.8792, GNorm = 6.4853, lr_0 = 9.6376e-04
Validation auc = 0.722535
Epoch 30
Validation auc = 0.664319
Epoch 31
Validation auc = 0.660563
Epoch 32
Validation auc = 0.658685
Epoch 33
Validation auc = 0.647887
Epoch 34
Loss = 5.8075e-01, PNorm = 40.9269, GNorm = 3.7780, lr_0 = 9.5552e-04
Validation auc = 0.654460
Epoch 35
Validation auc = 0.639437
Epoch 36
Validation auc = 0.624413
Epoch 37
Validation auc = 0.617371
Epoch 38
Validation auc = 0.606573
Epoch 39
Loss = 6.7670e-01, PNorm = 40.9526, GNorm = 5.6055, lr_0 = 9.4735e-04
Validation auc = 0.609390
Epoch 40
Validation auc = 0.616901
Epoch 41
Validation auc = 0.643192
Epoch 42
Validation auc = 0.664789
Epoch 43
Validation auc = 0.661972
Epoch 44
Loss = 6.2557e-01, PNorm = 40.9860, GNorm = 3.8241, lr_0 = 9.3925e-04
Validation auc = 0.649765
Epoch 45
Validation auc = 0.641315
Epoch 46
Validation auc = 0.636150
Epoch 47
Validation auc = 0.642723
Epoch 48
Validation auc = 0.644131
Epoch 49
Loss = 5.5161e-01, PNorm = 41.0275, GNorm = 3.8299, lr_0 = 9.3122e-04
Validation auc = 0.644601
Epoch 50
Validation auc = 0.640376
Epoch 51
Validation auc = 0.643192
Epoch 52
Validation auc = 0.649296
Epoch 53
Validation auc = 0.645540
Epoch 54
Loss = 6.3878e-01, PNorm = 41.0698, GNorm = 5.2715, lr_0 = 9.2326e-04
Validation auc = 0.638028
Epoch 55
Validation auc = 0.634272
Epoch 56
Validation auc = 0.642723
Epoch 57
Validation auc = 0.644131
Epoch 58
Validation auc = 0.650235
Epoch 59
Loss = 6.4173e-01, PNorm = 41.1079, GNorm = 3.5508, lr_0 = 9.1537e-04
Validation auc = 0.684507
Epoch 60
Validation auc = 0.685915
Epoch 61
Validation auc = 0.673239
Epoch 62
Validation auc = 0.672770
Epoch 63
Validation auc = 0.682160
Epoch 64
Loss = 5.6578e-01, PNorm = 41.1633, GNorm = 4.0758, lr_0 = 9.0754e-04
Validation auc = 0.687793
Epoch 65
Validation auc = 0.680282
Epoch 66
Validation auc = 0.663850
Epoch 67
Validation auc = 0.649765
Epoch 68
Validation auc = 0.642723
Epoch 69
Loss = 5.6712e-01, PNorm = 41.2104, GNorm = 3.5649, lr_0 = 8.9978e-04
Validation auc = 0.642254
Epoch 70
Validation auc = 0.647418
Epoch 71
Validation auc = 0.648357
Epoch 72
Validation auc = 0.644601
Epoch 73
Validation auc = 0.651174
Epoch 74
Loss = 5.9027e-01, PNorm = 41.2907, GNorm = 3.5333, lr_0 = 8.9209e-04
Validation auc = 0.653052
Epoch 75
Validation auc = 0.652582
Epoch 76
Validation auc = 0.661502
Epoch 77
Validation auc = 0.674178
Epoch 78
Validation auc = 0.690141
Epoch 79
Loss = 5.9295e-01, PNorm = 41.3316, GNorm = 4.2430, lr_0 = 8.8447e-04
Validation auc = 0.691549
Epoch 80
Validation auc = 0.694836
Epoch 81
Validation auc = 0.668545
Epoch 82
Validation auc = 0.638498
Epoch 83
Validation auc = 0.643662
Epoch 84
Loss = 5.5442e-01, PNorm = 41.4069, GNorm = 5.3977, lr_0 = 8.7691e-04
Validation auc = 0.655869
Epoch 85
Validation auc = 0.662441
Epoch 86
Validation auc = 0.666667
Epoch 87
Validation auc = 0.666197
Epoch 88
Validation auc = 0.661033
Epoch 89
Loss = 4.4261e-01, PNorm = 41.4816, GNorm = 3.2324, lr_0 = 8.6941e-04
Validation auc = 0.651643
Epoch 90
Validation auc = 0.640845
Epoch 91
Validation auc = 0.631925
Epoch 92
Validation auc = 0.615962
Epoch 93
Validation auc = 0.615962
Epoch 94
Loss = 4.8953e-01, PNorm = 41.5556, GNorm = 3.5319, lr_0 = 8.6198e-04
Validation auc = 0.620188
Epoch 95
Validation auc = 0.621596
Epoch 96
Validation auc = 0.619718
Epoch 97
Validation auc = 0.608451
Epoch 98
Validation auc = 0.606573
Epoch 99
Loss = 5.8039e-01, PNorm = 41.6272, GNorm = 3.8166, lr_0 = 8.5461e-04
Validation auc = 0.619249
Epoch 100
Validation auc = 0.626761
Epoch 101
Validation auc = 0.645070
Epoch 102
Validation auc = 0.664319
Epoch 103
Validation auc = 0.664789
Epoch 104
Loss = 4.5526e-01, PNorm = 41.7101, GNorm = 2.8701, lr_0 = 8.4730e-04
Validation auc = 0.666197
Epoch 105
Validation auc = 0.659155
Epoch 106
Validation auc = 0.644131
Epoch 107
Validation auc = 0.635211
Epoch 108
Validation auc = 0.638498
Epoch 109
Loss = 4.9369e-01, PNorm = 41.7676, GNorm = 2.2677, lr_0 = 8.4006e-04
Validation auc = 0.644131
Epoch 110
Validation auc = 0.650704
Epoch 111
Validation auc = 0.649765
Epoch 112
Validation auc = 0.638498
Epoch 113
Validation auc = 0.643192
Epoch 114
Loss = 4.8779e-01, PNorm = 41.8602, GNorm = 3.7279, lr_0 = 8.3288e-04
Validation auc = 0.652113
Epoch 115
Validation auc = 0.669014
Epoch 116
Validation auc = 0.663380
Epoch 117
Validation auc = 0.668545
Epoch 118
Validation auc = 0.668075
Epoch 119
Loss = 5.8264e-01, PNorm = 41.9553, GNorm = 2.3273, lr_0 = 8.2576e-04
Validation auc = 0.661033
Epoch 120
Validation auc = 0.660563
Epoch 121
Validation auc = 0.671362
Epoch 122
Validation auc = 0.676056
Epoch 123
Validation auc = 0.680282
Epoch 124
Loss = 4.0334e-01, PNorm = 42.0540, GNorm = 3.0817, lr_0 = 8.1870e-04
Validation auc = 0.671831
Epoch 125
Validation auc = 0.668545
Epoch 126
Validation auc = 0.666667
Epoch 127
Validation auc = 0.673239
Epoch 128
Validation auc = 0.670892
Epoch 129
Loss = 4.8197e-01, PNorm = 42.1586, GNorm = 2.7118, lr_0 = 8.1170e-04
Validation auc = 0.683099
Epoch 130
Validation auc = 0.700939
Epoch 131
Validation auc = 0.701408
Epoch 132
Validation auc = 0.700000
Epoch 133
Validation auc = 0.701878
Epoch 134
Loss = 3.7057e-01, PNorm = 42.2632, GNorm = 3.0221, lr_0 = 8.0476e-04
Validation auc = 0.706573
Epoch 135
Validation auc = 0.708920
Epoch 136
Validation auc = 0.703286
Epoch 137
Validation auc = 0.708920
Epoch 138
Validation auc = 0.713615
Epoch 139
Loss = 4.2169e-01, PNorm = 42.3812, GNorm = 2.1885, lr_0 = 7.9788e-04
Validation auc = 0.709390
Epoch 140
Validation auc = 0.707042
Epoch 141
Validation auc = 0.700000
Epoch 142
Validation auc = 0.701408
Epoch 143
Validation auc = 0.704225
Epoch 144
Loss = 3.9695e-01, PNorm = 42.4665, GNorm = 2.3187, lr_0 = 7.9106e-04
Validation auc = 0.709859
Epoch 145
Validation auc = 0.714554
Epoch 146
Validation auc = 0.710798
Epoch 147
Validation auc = 0.716901
Epoch 148
Validation auc = 0.714554
Epoch 149
Loss = 4.3793e-01, PNorm = 42.5498, GNorm = 3.9545, lr_0 = 7.8430e-04
Validation auc = 0.723474
Epoch 150
Validation auc = 0.728169
Epoch 151
Validation auc = 0.734742
Epoch 152
Validation auc = 0.738967
Epoch 153
Validation auc = 0.738498
Epoch 154
Loss = 4.4117e-01, PNorm = 42.6555, GNorm = 2.8707, lr_0 = 7.7759e-04
Validation auc = 0.727700
Epoch 155
Validation auc = 0.724883
Epoch 156
Validation auc = 0.716432
Epoch 157
Validation auc = 0.714554
Epoch 158
Validation auc = 0.703286
Epoch 159
Loss = 4.5156e-01, PNorm = 42.7668, GNorm = 3.4066, lr_0 = 7.7095e-04
Validation auc = 0.707042
Epoch 160
Validation auc = 0.721596
Epoch 161
Validation auc = 0.730047
Epoch 162
Validation auc = 0.734742
Epoch 163
Validation auc = 0.735681
Epoch 164
Loss = 4.3522e-01, PNorm = 42.8561, GNorm = 3.6832, lr_0 = 7.6436e-04
Validation auc = 0.714085
Epoch 165
Validation auc = 0.702817
Epoch 166
Validation auc = 0.703286
Epoch 167
Validation auc = 0.693897
Epoch 168
Validation auc = 0.693427
Epoch 169
Loss = 3.3454e-01, PNorm = 42.9635, GNorm = 2.6490, lr_0 = 7.5782e-04
Validation auc = 0.694836
Epoch 170
Validation auc = 0.697183
Epoch 171
Validation auc = 0.704695
Epoch 172
Validation auc = 0.713146
Epoch 173
Validation auc = 0.713146
Epoch 174
Loss = 3.3167e-01, PNorm = 43.0684, GNorm = 1.9595, lr_0 = 7.5134e-04
Validation auc = 0.719718
Epoch 175
Validation auc = 0.718310
Epoch 176
Validation auc = 0.714085
Epoch 177
Validation auc = 0.713146
Epoch 178
Validation auc = 0.712207
Epoch 179
Loss = 3.0406e-01, PNorm = 43.1787, GNorm = 2.6592, lr_0 = 7.4492e-04
Validation auc = 0.728638
Epoch 180
Validation auc = 0.746948
Epoch 181
Validation auc = 0.751643
Epoch 182
Validation auc = 0.740376
Epoch 183
Validation auc = 0.738967
Epoch 184
Loss = 4.1302e-01, PNorm = 43.2710, GNorm = 2.9524, lr_0 = 7.3855e-04
Validation auc = 0.746479
Epoch 185
Validation auc = 0.748826
Epoch 186
Validation auc = 0.740376
Epoch 187
Validation auc = 0.725822
Epoch 188
Validation auc = 0.705634
Epoch 189
Loss = 4.0835e-01, PNorm = 43.3506, GNorm = 2.3178, lr_0 = 7.3224e-04
Validation auc = 0.715493
Epoch 190
Validation auc = 0.719718
Epoch 191
Validation auc = 0.719718
Epoch 192
Validation auc = 0.724883
Epoch 193
Validation auc = 0.724883
Epoch 194
Loss = 3.8194e-01, PNorm = 43.4408, GNorm = 1.8036, lr_0 = 7.2598e-04
Validation auc = 0.723944
Epoch 195
Validation auc = 0.718310
Epoch 196
Validation auc = 0.726761
Epoch 197
Validation auc = 0.726291
Epoch 198
Validation auc = 0.729577
Epoch 199
Loss = 3.3246e-01, PNorm = 43.5347, GNorm = 2.2120, lr_0 = 7.1977e-04
Validation auc = 0.719718
Epoch 200
Validation auc = 0.710798
Epoch 201
Validation auc = 0.699531
Epoch 202
Validation auc = 0.708451
Epoch 203
Validation auc = 0.709859
Epoch 204
Loss = 3.4608e-01, PNorm = 43.6281, GNorm = 3.8036, lr_0 = 7.1362e-04
Validation auc = 0.719249
Epoch 205
Validation auc = 0.724883
Epoch 206
Validation auc = 0.726291
Epoch 207
Validation auc = 0.726291
Epoch 208
Validation auc = 0.727230
Epoch 209
Loss = 3.7296e-01, PNorm = 43.7045, GNorm = 2.8805, lr_0 = 7.0752e-04
Validation auc = 0.727700
Epoch 210
Validation auc = 0.725352
Epoch 211
Validation auc = 0.735211
Epoch 212
Validation auc = 0.730986
Epoch 213
Validation auc = 0.724413
Epoch 214
Loss = 3.0332e-01, PNorm = 43.8134, GNorm = 1.9838, lr_0 = 7.0147e-04
Validation auc = 0.725352
Epoch 215
Validation auc = 0.721596
Epoch 216
Validation auc = 0.724883
Epoch 217
Validation auc = 0.720657
Epoch 218
Validation auc = 0.724413
Epoch 219
Loss = 3.6710e-01, PNorm = 43.9164, GNorm = 3.2902, lr_0 = 6.9548e-04
Validation auc = 0.720657
Epoch 220
Validation auc = 0.725822
Epoch 221
Validation auc = 0.725822
Epoch 222
Validation auc = 0.717371
Epoch 223
Validation auc = 0.719249
Epoch 224
Loss = 3.0545e-01, PNorm = 43.9850, GNorm = 2.4454, lr_0 = 6.8953e-04
Validation auc = 0.722535
Epoch 225
Validation auc = 0.718779
Epoch 226
Validation auc = 0.720657
Epoch 227
Validation auc = 0.721127
Epoch 228
Validation auc = 0.722066
Epoch 229
Loss = 2.8623e-01, PNorm = 44.0787, GNorm = 2.3642, lr_0 = 6.8364e-04
Validation auc = 0.725352
Epoch 230
Validation auc = 0.726291
Epoch 231
Validation auc = 0.723944
Epoch 232
Validation auc = 0.727230
Epoch 233
Validation auc = 0.727230
Epoch 234
Loss = 3.4498e-01, PNorm = 44.1853, GNorm = 4.1308, lr_0 = 6.7779e-04
Validation auc = 0.724883
Epoch 235
Validation auc = 0.720188
Epoch 236
Validation auc = 0.717371
Epoch 237
Validation auc = 0.715023
Epoch 238
Validation auc = 0.715493
Epoch 239
Loss = 2.1002e-01, PNorm = 44.2865, GNorm = 2.1728, lr_0 = 6.7200e-04
Validation auc = 0.717840
Epoch 240
Validation auc = 0.725352
Epoch 241
Validation auc = 0.729108
Epoch 242
Validation auc = 0.732864
Epoch 243
Validation auc = 0.736150
Epoch 244
Loss = 2.1420e-01, PNorm = 44.4062, GNorm = 2.2416, lr_0 = 6.6625e-04
Validation auc = 0.736150
Epoch 245
Validation auc = 0.737089
Epoch 246
Validation auc = 0.737089
Epoch 247
Validation auc = 0.743662
Epoch 248
Validation auc = 0.743662
Epoch 249
Loss = 2.1093e-01, PNorm = 44.4928, GNorm = 1.9247, lr_0 = 6.6056e-04
Validation auc = 0.741315
Epoch 250
Validation auc = 0.742254
Epoch 251
Validation auc = 0.745070
Epoch 252
Validation auc = 0.739437
Epoch 253
Validation auc = 0.741784
Epoch 254
Loss = 2.4774e-01, PNorm = 44.5642, GNorm = 4.3796, lr_0 = 6.5491e-04
Validation auc = 0.740845
Epoch 255
Validation auc = 0.737559
Epoch 256
Validation auc = 0.732394
Epoch 257
Validation auc = 0.729577
Epoch 258
Validation auc = 0.727230
Epoch 259
Loss = 2.6047e-01, PNorm = 44.6649, GNorm = 2.1701, lr_0 = 6.4931e-04
Validation auc = 0.730516
Epoch 260
Validation auc = 0.736620
Epoch 261
Validation auc = 0.732864
Epoch 262
Validation auc = 0.729577
Epoch 263
Validation auc = 0.727700
Epoch 264
Loss = 1.8792e-01, PNorm = 44.7490, GNorm = 2.5597, lr_0 = 6.4376e-04
Validation auc = 0.728638
Epoch 265
Validation auc = 0.723944
Epoch 266
Validation auc = 0.718310
Epoch 267
Validation auc = 0.711268
Epoch 268
Validation auc = 0.709390
Epoch 269
Loss = 3.5221e-01, PNorm = 44.8222, GNorm = 2.6000, lr_0 = 6.3826e-04
Validation auc = 0.708451
Epoch 270
Validation auc = 0.709859
Epoch 271
Validation auc = 0.710798
Epoch 272
Validation auc = 0.708920
Epoch 273
Validation auc = 0.704225
Epoch 274
Loss = 3.1966e-01, PNorm = 44.9041, GNorm = 4.6139, lr_0 = 6.3280e-04
Validation auc = 0.707512
Epoch 275
Validation auc = 0.706103
Epoch 276
Validation auc = 0.708451
Epoch 277
Validation auc = 0.710798
Epoch 278
Validation auc = 0.716432
Epoch 279
Loss = 2.2695e-01, PNorm = 44.9770, GNorm = 1.6474, lr_0 = 6.2739e-04
Validation auc = 0.720188
Epoch 280
Validation auc = 0.722066
Epoch 281
Validation auc = 0.727230
Epoch 282
Validation auc = 0.731925
Epoch 283
Validation auc = 0.739437
Epoch 284
Loss = 2.6159e-01, PNorm = 45.0653, GNorm = 3.8780, lr_0 = 6.2203e-04
Validation auc = 0.744131
Epoch 285
Validation auc = 0.748357
Epoch 286
Validation auc = 0.747887
Epoch 287
Validation auc = 0.743662
Epoch 288
Validation auc = 0.739437
Epoch 289
Loss = 2.6213e-01, PNorm = 45.1488, GNorm = 2.3032, lr_0 = 6.1671e-04
Validation auc = 0.741315
Epoch 290
Validation auc = 0.737559
Epoch 291
Validation auc = 0.738028
Epoch 292
Validation auc = 0.736620
Epoch 293
Validation auc = 0.733803
Epoch 294
Loss = 1.8566e-01, PNorm = 45.2323, GNorm = 3.1061, lr_0 = 6.1144e-04
Validation auc = 0.732394
Epoch 295
Validation auc = 0.726291
Epoch 296
Validation auc = 0.720188
Epoch 297
Validation auc = 0.723944
Epoch 298
Validation auc = 0.727700
Epoch 299
Loss = 1.8340e-01, PNorm = 45.3164, GNorm = 2.9613, lr_0 = 6.0621e-04
Validation auc = 0.732394
Model 0 best validation auc = 0.751643 on epoch 181
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.918724
Ensemble test auc = 0.918724
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.603286
Epoch 1
Validation auc = 0.682629
Epoch 2
Validation auc = 0.666667
Epoch 3
Validation auc = 0.664789
Epoch 4
Loss = 1.1578e+00, PNorm = 40.6675, GNorm = 18.6631, lr_0 = 6.5000e-04
Validation auc = 0.663850
Epoch 5
Validation auc = 0.662911
Epoch 6
Validation auc = 0.654930
Epoch 7
Validation auc = 0.643192
Epoch 8
Validation auc = 0.621127
Epoch 9
Loss = 8.0703e-01, PNorm = 40.7168, GNorm = 6.9227, lr_0 = 9.9743e-04
Validation auc = 0.627700
Epoch 10
Validation auc = 0.630986
Epoch 11
Validation auc = 0.641315
Epoch 12
Validation auc = 0.638967
Epoch 13
Validation auc = 0.645070
Epoch 14
Loss = 6.0281e-01, PNorm = 40.7580, GNorm = 7.3325, lr_0 = 9.8890e-04
Validation auc = 0.664789
Epoch 15
Validation auc = 0.675117
Epoch 16
Validation auc = 0.676526
Epoch 17
Validation auc = 0.675587
Epoch 18
Validation auc = 0.676526
Epoch 19
Loss = 6.3585e-01, PNorm = 40.8023, GNorm = 6.2515, lr_0 = 9.8045e-04
Validation auc = 0.665258
Epoch 20
Validation auc = 0.649296
Epoch 21
Validation auc = 0.661033
Epoch 22
Validation auc = 0.667606
Epoch 23
Validation auc = 0.688263
Epoch 24
Loss = 7.1434e-01, PNorm = 40.8367, GNorm = 5.4259, lr_0 = 9.7207e-04
Validation auc = 0.714085
Epoch 25
Validation auc = 0.705634
Epoch 26
Validation auc = 0.686854
Epoch 27
Validation auc = 0.686854
Epoch 28
Validation auc = 0.686385
Epoch 29
Loss = 6.4502e-01, PNorm = 40.8571, GNorm = 5.1571, lr_0 = 9.6376e-04
Validation auc = 0.682160
Epoch 30
Validation auc = 0.680282
Epoch 31
Validation auc = 0.690610
Epoch 32
Validation auc = 0.682160
Epoch 33
Validation auc = 0.674648
Epoch 34
Loss = 6.6455e-01, PNorm = 40.8930, GNorm = 5.7153, lr_0 = 9.5552e-04
Validation auc = 0.677934
Epoch 35
Validation auc = 0.678404
Epoch 36
Validation auc = 0.669953
Epoch 37
Validation auc = 0.661502
Epoch 38
Validation auc = 0.668075
Epoch 39
Loss = 6.6550e-01, PNorm = 40.9279, GNorm = 4.4305, lr_0 = 9.4735e-04
Validation auc = 0.684977
Epoch 40
Validation auc = 0.689671
Epoch 41
Validation auc = 0.688732
Epoch 42
Validation auc = 0.680282
Epoch 43
Validation auc = 0.679343
Epoch 44
Loss = 5.7585e-01, PNorm = 40.9649, GNorm = 4.5449, lr_0 = 9.3925e-04
Validation auc = 0.685915
Epoch 45
Validation auc = 0.683568
Epoch 46
Validation auc = 0.692958
Epoch 47
Validation auc = 0.697653
Epoch 48
Validation auc = 0.692488
Epoch 49
Loss = 5.6212e-01, PNorm = 41.0062, GNorm = 3.7860, lr_0 = 9.3122e-04
Validation auc = 0.689202
Epoch 50
Validation auc = 0.680751
Epoch 51
Validation auc = 0.679343
Epoch 52
Validation auc = 0.682160
Epoch 53
Validation auc = 0.686854
Epoch 54
Loss = 6.0842e-01, PNorm = 41.0402, GNorm = 4.9117, lr_0 = 9.2326e-04
Validation auc = 0.683568
Epoch 55
Validation auc = 0.670892
Epoch 56
Validation auc = 0.656808
Epoch 57
Validation auc = 0.656338
Epoch 58
Validation auc = 0.648826
Epoch 59
Loss = 5.3344e-01, PNorm = 41.0928, GNorm = 2.5048, lr_0 = 9.1537e-04
Validation auc = 0.653521
Epoch 60
Validation auc = 0.644601
Epoch 61
Validation auc = 0.646009
Epoch 62
Validation auc = 0.648826
Epoch 63
Validation auc = 0.648826
Epoch 64
Loss = 5.5027e-01, PNorm = 41.1510, GNorm = 2.9258, lr_0 = 9.0754e-04
Validation auc = 0.646479
Epoch 65
Validation auc = 0.638967
Epoch 66
Validation auc = 0.647418
Epoch 67
Validation auc = 0.647887
Epoch 68
Validation auc = 0.647887
Epoch 69
Loss = 5.2007e-01, PNorm = 41.2037, GNorm = 2.7923, lr_0 = 8.9978e-04
Validation auc = 0.641784
Epoch 70
Validation auc = 0.634272
Epoch 71
Validation auc = 0.629577
Epoch 72
Validation auc = 0.631925
Epoch 73
Validation auc = 0.637089
Epoch 74
Loss = 5.7075e-01, PNorm = 41.2577, GNorm = 3.4166, lr_0 = 8.9209e-04
Validation auc = 0.640845
Epoch 75
Validation auc = 0.640376
Epoch 76
Validation auc = 0.639437
Epoch 77
Validation auc = 0.640376
Epoch 78
Validation auc = 0.642254
Epoch 79
Loss = 5.1405e-01, PNorm = 41.3365, GNorm = 3.2384, lr_0 = 8.8447e-04
Validation auc = 0.640845
Epoch 80
Validation auc = 0.633803
Epoch 81
Validation auc = 0.636150
Epoch 82
Validation auc = 0.639437
Epoch 83
Validation auc = 0.647887
Epoch 84
Loss = 4.8568e-01, PNorm = 41.3885, GNorm = 2.7124, lr_0 = 8.7691e-04
Validation auc = 0.653521
Epoch 85
Validation auc = 0.655399
Epoch 86
Validation auc = 0.655399
Epoch 87
Validation auc = 0.655399
Epoch 88
Validation auc = 0.658216
Epoch 89
Loss = 4.8943e-01, PNorm = 41.4778, GNorm = 2.8749, lr_0 = 8.6941e-04
Validation auc = 0.671831
Epoch 90
Validation auc = 0.673709
Epoch 91
Validation auc = 0.652113
Epoch 92
Validation auc = 0.648826
Epoch 93
Validation auc = 0.640376
Epoch 94
Loss = 5.6406e-01, PNorm = 41.5380, GNorm = 2.6476, lr_0 = 8.6198e-04
Validation auc = 0.642254
Epoch 95
Validation auc = 0.649765
Epoch 96
Validation auc = 0.660563
Epoch 97
Validation auc = 0.656808
Epoch 98
Validation auc = 0.656338
Epoch 99
Loss = 5.2838e-01, PNorm = 41.6108, GNorm = 3.2421, lr_0 = 8.5461e-04
Validation auc = 0.672770
Epoch 100
Validation auc = 0.680751
Epoch 101
Validation auc = 0.678873
Epoch 102
Validation auc = 0.682160
Epoch 103
Validation auc = 0.683568
Epoch 104
Loss = 5.5900e-01, PNorm = 41.6980, GNorm = 3.9225, lr_0 = 8.4730e-04
Validation auc = 0.685915
Epoch 105
Validation auc = 0.686385
Epoch 106
Validation auc = 0.690141
Epoch 107
Validation auc = 0.708451
Epoch 108
Validation auc = 0.730516
Epoch 109
Loss = 4.6490e-01, PNorm = 41.7674, GNorm = 3.6123, lr_0 = 8.4006e-04
Validation auc = 0.726291
Epoch 110
Validation auc = 0.697183
Epoch 111
Validation auc = 0.676995
Epoch 112
Validation auc = 0.665728
Epoch 113
Validation auc = 0.670892
Epoch 114
Loss = 5.2028e-01, PNorm = 41.8708, GNorm = 3.4822, lr_0 = 8.3288e-04
Validation auc = 0.687793
Epoch 115
Validation auc = 0.706573
Epoch 116
Validation auc = 0.697183
Epoch 117
Validation auc = 0.689671
Epoch 118
Validation auc = 0.688732
Epoch 119
Loss = 4.5549e-01, PNorm = 41.9655, GNorm = 2.7803, lr_0 = 8.2576e-04
Validation auc = 0.684038
Epoch 120
Validation auc = 0.670892
Epoch 121
Validation auc = 0.671831
Epoch 122
Validation auc = 0.669014
Epoch 123
Validation auc = 0.674178
Epoch 124
Loss = 4.3298e-01, PNorm = 42.0772, GNorm = 2.1209, lr_0 = 8.1870e-04
Validation auc = 0.679343
Epoch 125
Validation auc = 0.677465
Epoch 126
Validation auc = 0.681221
Epoch 127
Validation auc = 0.690610
Epoch 128
Validation auc = 0.687793
Epoch 129
Loss = 4.8987e-01, PNorm = 42.1762, GNorm = 4.0184, lr_0 = 8.1170e-04
Validation auc = 0.683099
Epoch 130
Validation auc = 0.683568
Epoch 131
Validation auc = 0.682629
Epoch 132
Validation auc = 0.690610
Epoch 133
Validation auc = 0.696714
Epoch 134
Loss = 3.9693e-01, PNorm = 42.2874, GNorm = 3.0544, lr_0 = 8.0476e-04
Validation auc = 0.698122
Epoch 135
Validation auc = 0.699531
Epoch 136
Validation auc = 0.699531
Epoch 137
Validation auc = 0.702347
Epoch 138
Validation auc = 0.707981
Epoch 139
Loss = 3.9907e-01, PNorm = 42.4010, GNorm = 2.3495, lr_0 = 7.9788e-04
Validation auc = 0.710329
Epoch 140
Validation auc = 0.719249
Epoch 141
Validation auc = 0.727230
Epoch 142
Validation auc = 0.718310
Epoch 143
Validation auc = 0.711268
Epoch 144
Loss = 4.4270e-01, PNorm = 42.5147, GNorm = 2.9390, lr_0 = 7.9106e-04
Validation auc = 0.701878
Epoch 145
Validation auc = 0.697653
Epoch 146
Validation auc = 0.698592
Epoch 147
Validation auc = 0.707512
Epoch 148
Validation auc = 0.711737
Epoch 149
Loss = 5.1841e-01, PNorm = 42.6306, GNorm = 3.3875, lr_0 = 7.8430e-04
Validation auc = 0.715962
Epoch 150
Validation auc = 0.716901
Epoch 151
Validation auc = 0.715962
Epoch 152
Validation auc = 0.716901
Epoch 153
Validation auc = 0.723474
Epoch 154
Loss = 3.8656e-01, PNorm = 42.7469, GNorm = 3.3456, lr_0 = 7.7759e-04
Validation auc = 0.723944
Epoch 155
Validation auc = 0.722066
Epoch 156
Validation auc = 0.725822
Epoch 157
Validation auc = 0.722535
Epoch 158
Validation auc = 0.724413
Epoch 159
Loss = 3.2216e-01, PNorm = 42.8652, GNorm = 2.8300, lr_0 = 7.7095e-04
Validation auc = 0.723005
Epoch 160
Validation auc = 0.719718
Epoch 161
Validation auc = 0.721596
Epoch 162
Validation auc = 0.719718
Epoch 163
Validation auc = 0.715493
Epoch 164
Loss = 3.6263e-01, PNorm = 42.9752, GNorm = 2.0726, lr_0 = 7.6436e-04
Validation auc = 0.717840
Epoch 165
Validation auc = 0.711737
Epoch 166
Validation auc = 0.710329
Epoch 167
Validation auc = 0.706103
Epoch 168
Validation auc = 0.710798
Epoch 169
Loss = 3.3312e-01, PNorm = 43.0876, GNorm = 1.4397, lr_0 = 7.5782e-04
Validation auc = 0.712207
Epoch 170
Validation auc = 0.710798
Epoch 171
Validation auc = 0.705164
Epoch 172
Validation auc = 0.702347
Epoch 173
Validation auc = 0.702817
Epoch 174
Loss = 3.9319e-01, PNorm = 43.1969, GNorm = 2.2667, lr_0 = 7.5134e-04
Validation auc = 0.707042
Epoch 175
Validation auc = 0.705634
Epoch 176
Validation auc = 0.709390
Epoch 177
Validation auc = 0.711268
Epoch 178
Validation auc = 0.712676
Epoch 179
Loss = 3.5268e-01, PNorm = 43.3053, GNorm = 2.1145, lr_0 = 7.4492e-04
Validation auc = 0.719718
Epoch 180
Validation auc = 0.720188
Epoch 181
Validation auc = 0.722535
Epoch 182
Validation auc = 0.719249
Epoch 183
Validation auc = 0.722535
Epoch 184
Loss = 4.0613e-01, PNorm = 43.4103, GNorm = 3.4378, lr_0 = 7.3855e-04
Validation auc = 0.726291
Epoch 185
Validation auc = 0.722535
Epoch 186
Validation auc = 0.717371
Epoch 187
Validation auc = 0.709859
Epoch 188
Validation auc = 0.707981
Epoch 189
Loss = 3.5514e-01, PNorm = 43.5181, GNorm = 2.6670, lr_0 = 7.3224e-04
Validation auc = 0.714085
Epoch 190
Validation auc = 0.719718
Epoch 191
Validation auc = 0.722535
Epoch 192
Validation auc = 0.735681
Epoch 193
Validation auc = 0.734272
Epoch 194
Loss = 3.9230e-01, PNorm = 43.6279, GNorm = 2.4198, lr_0 = 7.2598e-04
Validation auc = 0.724883
Epoch 195
Validation auc = 0.717371
Epoch 196
Validation auc = 0.715023
Epoch 197
Validation auc = 0.708451
Epoch 198
Validation auc = 0.707042
Epoch 199
Loss = 3.3719e-01, PNorm = 43.7318, GNorm = 1.8758, lr_0 = 7.1977e-04
Validation auc = 0.710329
Epoch 200
Validation auc = 0.706573
Epoch 201
Validation auc = 0.705164
Epoch 202
Validation auc = 0.708451
Epoch 203
Validation auc = 0.718310
Epoch 204
Loss = 3.5699e-01, PNorm = 43.8292, GNorm = 3.3207, lr_0 = 7.1362e-04
Validation auc = 0.726761
Epoch 205
Validation auc = 0.726761
Epoch 206
Validation auc = 0.726291
Epoch 207
Validation auc = 0.726761
Epoch 208
Validation auc = 0.724413
Epoch 209
Loss = 2.8859e-01, PNorm = 43.9209, GNorm = 1.9776, lr_0 = 7.0752e-04
Validation auc = 0.723474
Epoch 210
Validation auc = 0.718779
Epoch 211
Validation auc = 0.719718
Epoch 212
Validation auc = 0.725822
Epoch 213
Validation auc = 0.730986
Epoch 214
Loss = 3.6578e-01, PNorm = 44.0188, GNorm = 3.5555, lr_0 = 7.0147e-04
Validation auc = 0.726291
Epoch 215
Validation auc = 0.718779
Epoch 216
Validation auc = 0.721596
Epoch 217
Validation auc = 0.720657
Epoch 218
Validation auc = 0.723944
Epoch 219
Loss = 3.2868e-01, PNorm = 44.1161, GNorm = 2.0936, lr_0 = 6.9548e-04
Validation auc = 0.723474
Epoch 220
Validation auc = 0.724883
Epoch 221
Validation auc = 0.725822
Epoch 222
Validation auc = 0.728638
Epoch 223
Validation auc = 0.731455
Epoch 224
Loss = 4.2067e-01, PNorm = 44.2279, GNorm = 3.6196, lr_0 = 6.8953e-04
Validation auc = 0.725352
Epoch 225
Validation auc = 0.723944
Epoch 226
Validation auc = 0.725822
Epoch 227
Validation auc = 0.727700
Epoch 228
Validation auc = 0.730047
Epoch 229
Loss = 3.5825e-01, PNorm = 44.3240, GNorm = 3.9799, lr_0 = 6.8364e-04
Validation auc = 0.735211
Epoch 230
Validation auc = 0.732864
Epoch 231
Validation auc = 0.732864
Epoch 232
Validation auc = 0.733803
Epoch 233
Validation auc = 0.739437
Epoch 234
Loss = 3.2741e-01, PNorm = 44.4289, GNorm = 1.8655, lr_0 = 6.7779e-04
Validation auc = 0.738967
Epoch 235
Validation auc = 0.739437
Epoch 236
Validation auc = 0.737559
Epoch 237
Validation auc = 0.734742
Epoch 238
Validation auc = 0.737089
Epoch 239
Loss = 3.4168e-01, PNorm = 44.5322, GNorm = 2.8227, lr_0 = 6.7200e-04
Validation auc = 0.734742
Epoch 240
Validation auc = 0.738028
Epoch 241
Validation auc = 0.738498
Epoch 242
Validation auc = 0.737089
Epoch 243
Validation auc = 0.733803
Epoch 244
Loss = 2.7933e-01, PNorm = 44.6158, GNorm = 2.6784, lr_0 = 6.6625e-04
Validation auc = 0.731455
Epoch 245
Validation auc = 0.732394
Epoch 246
Validation auc = 0.728638
Epoch 247
Validation auc = 0.728638
Epoch 248
Validation auc = 0.727230
Epoch 249
Loss = 3.2775e-01, PNorm = 44.6892, GNorm = 7.4501, lr_0 = 6.6056e-04
Validation auc = 0.729108
Epoch 250
Validation auc = 0.729108
Epoch 251
Validation auc = 0.730986
Epoch 252
Validation auc = 0.729577
Epoch 253
Validation auc = 0.730986
Epoch 254
Loss = 2.8893e-01, PNorm = 44.7835, GNorm = 3.2858, lr_0 = 6.5491e-04
Validation auc = 0.731925
Epoch 255
Validation auc = 0.732864
Epoch 256
Validation auc = 0.735681
Epoch 257
Validation auc = 0.735211
Epoch 258
Validation auc = 0.736620
Epoch 259
Loss = 3.0361e-01, PNorm = 44.8716, GNorm = 2.5285, lr_0 = 6.4931e-04
Validation auc = 0.739906
Epoch 260
Validation auc = 0.741784
Epoch 261
Validation auc = 0.738028
Epoch 262
Validation auc = 0.738498
Epoch 263
Validation auc = 0.739906
Epoch 264
Loss = 3.1136e-01, PNorm = 44.9499, GNorm = 2.4712, lr_0 = 6.4376e-04
Validation auc = 0.746948
Epoch 265
Validation auc = 0.749296
Epoch 266
Validation auc = 0.744601
Epoch 267
Validation auc = 0.744131
Epoch 268
Validation auc = 0.736620
Epoch 269
Loss = 2.5056e-01, PNorm = 45.0418, GNorm = 3.1355, lr_0 = 6.3826e-04
Validation auc = 0.735211
Epoch 270
Validation auc = 0.723944
Epoch 271
Validation auc = 0.719718
Epoch 272
Validation auc = 0.721127
Epoch 273
Validation auc = 0.722066
Epoch 274
Loss = 2.6467e-01, PNorm = 45.1052, GNorm = 2.6963, lr_0 = 6.3280e-04
Validation auc = 0.726291
Epoch 275
Validation auc = 0.727230
Epoch 276
Validation auc = 0.729577
Epoch 277
Validation auc = 0.725352
Epoch 278
Validation auc = 0.727700
Epoch 279
Loss = 2.8762e-01, PNorm = 45.1775, GNorm = 1.6946, lr_0 = 6.2739e-04
Validation auc = 0.726291
Epoch 280
Validation auc = 0.729577
Epoch 281
Validation auc = 0.731925
Epoch 282
Validation auc = 0.730986
Epoch 283
Validation auc = 0.732394
Epoch 284
Loss = 2.6083e-01, PNorm = 45.2759, GNorm = 2.3677, lr_0 = 6.2203e-04
Validation auc = 0.732864
Epoch 285
Validation auc = 0.739437
Epoch 286
Validation auc = 0.738967
Epoch 287
Validation auc = 0.735681
Epoch 288
Validation auc = 0.723005
Epoch 289
Loss = 1.8965e-01, PNorm = 45.3599, GNorm = 2.6017, lr_0 = 6.1671e-04
Validation auc = 0.715962
Epoch 290
Validation auc = 0.714554
Epoch 291
Validation auc = 0.721596
Epoch 292
Validation auc = 0.726291
Epoch 293
Validation auc = 0.730047
Epoch 294
Loss = 1.6455e-01, PNorm = 45.4562, GNorm = 1.4983, lr_0 = 6.1144e-04
Validation auc = 0.732394
Epoch 295
Validation auc = 0.742254
Epoch 296
Validation auc = 0.745540
Epoch 297
Validation auc = 0.741784
Epoch 298
Validation auc = 0.735681
Epoch 299
Loss = 1.6366e-01, PNorm = 45.5460, GNorm = 1.6271, lr_0 = 6.0621e-04
Validation auc = 0.734742
Model 0 best validation auc = 0.749296 on epoch 265
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.919326
Ensemble test auc = 0.919326
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.536150
Epoch 1
Validation auc = 0.675117
Epoch 2
Validation auc = 0.665728
Epoch 3
Validation auc = 0.670892
Epoch 4
Loss = 1.1533e+00, PNorm = 40.6663, GNorm = 16.0173, lr_0 = 6.5000e-04
Validation auc = 0.692488
Epoch 5
Validation auc = 0.704695
Epoch 6
Validation auc = 0.703756
Epoch 7
Validation auc = 0.704695
Epoch 8
Validation auc = 0.712676
Epoch 9
Loss = 8.8233e-01, PNorm = 40.7134, GNorm = 9.5232, lr_0 = 9.9743e-04
Validation auc = 0.735681
Epoch 10
Validation auc = 0.696244
Epoch 11
Validation auc = 0.669953
Epoch 12
Validation auc = 0.666197
Epoch 13
Validation auc = 0.657746
Epoch 14
Loss = 8.9626e-01, PNorm = 40.7581, GNorm = 7.0457, lr_0 = 9.8890e-04
Validation auc = 0.694836
Epoch 15
Validation auc = 0.670423
Epoch 16
Validation auc = 0.659155
Epoch 17
Validation auc = 0.658685
Epoch 18
Validation auc = 0.651643
Epoch 19
Loss = 7.1981e-01, PNorm = 40.8034, GNorm = 6.7761, lr_0 = 9.8045e-04
Validation auc = 0.634272
Epoch 20
Validation auc = 0.637089
Epoch 21
Validation auc = 0.633333
Epoch 22
Validation auc = 0.642723
Epoch 23
Validation auc = 0.650704
Epoch 24
Loss = 6.1751e-01, PNorm = 40.8444, GNorm = 7.1741, lr_0 = 9.7207e-04
Validation auc = 0.648357
Epoch 25
Validation auc = 0.661972
Epoch 26
Validation auc = 0.672300
Epoch 27
Validation auc = 0.684507
Epoch 28
Validation auc = 0.700469
Epoch 29
Loss = 5.6580e-01, PNorm = 40.8728, GNorm = 4.9620, lr_0 = 9.6376e-04
Validation auc = 0.667136
Epoch 30
Validation auc = 0.663850
Epoch 31
Validation auc = 0.666197
Epoch 32
Validation auc = 0.674648
Epoch 33
Validation auc = 0.680751
Epoch 34
Loss = 7.1216e-01, PNorm = 40.9066, GNorm = 6.3080, lr_0 = 9.5552e-04
Validation auc = 0.692958
Epoch 35
Validation auc = 0.689202
Epoch 36
Validation auc = 0.685915
Epoch 37
Validation auc = 0.680282
Epoch 38
Validation auc = 0.677465
Epoch 39
Loss = 6.6779e-01, PNorm = 40.9509, GNorm = 4.4193, lr_0 = 9.4735e-04
Validation auc = 0.665728
Epoch 40
Validation auc = 0.642723
Epoch 41
Validation auc = 0.618779
Epoch 42
Validation auc = 0.630516
Epoch 43
Validation auc = 0.644131
Epoch 44
Loss = 5.5403e-01, PNorm = 40.9773, GNorm = 5.3221, lr_0 = 9.3925e-04
Validation auc = 0.642254
Epoch 45
Validation auc = 0.637559
Epoch 46
Validation auc = 0.624413
Epoch 47
Validation auc = 0.667606
Epoch 48
Validation auc = 0.682629
Epoch 49
Loss = 5.5389e-01, PNorm = 41.0108, GNorm = 3.2024, lr_0 = 9.3122e-04
Validation auc = 0.669014
Epoch 50
Validation auc = 0.653521
Epoch 51
Validation auc = 0.644131
Epoch 52
Validation auc = 0.646009
Epoch 53
Validation auc = 0.654460
Epoch 54
Loss = 5.4156e-01, PNorm = 41.0342, GNorm = 3.5037, lr_0 = 9.2326e-04
Validation auc = 0.653521
Epoch 55
Validation auc = 0.655869
Epoch 56
Validation auc = 0.650235
Epoch 57
Validation auc = 0.655869
Epoch 58
Validation auc = 0.652113
Epoch 59
Loss = 5.5974e-01, PNorm = 41.0695, GNorm = 4.2071, lr_0 = 9.1537e-04
Validation auc = 0.646009
Epoch 60
Validation auc = 0.644601
Epoch 61
Validation auc = 0.639437
Epoch 62
Validation auc = 0.637089
Epoch 63
Validation auc = 0.649765
Epoch 64
Loss = 5.7852e-01, PNorm = 41.1212, GNorm = 3.7879, lr_0 = 9.0754e-04
Validation auc = 0.659155
Epoch 65
Validation auc = 0.659624
Epoch 66
Validation auc = 0.657746
Epoch 67
Validation auc = 0.657746
Epoch 68
Validation auc = 0.655399
Epoch 69
Loss = 5.3268e-01, PNorm = 41.1519, GNorm = 3.4951, lr_0 = 8.9978e-04
Validation auc = 0.644601
Epoch 70
Validation auc = 0.642254
Epoch 71
Validation auc = 0.644131
Epoch 72
Validation auc = 0.649296
Epoch 73
Validation auc = 0.650235
Epoch 74
Loss = 6.3504e-01, PNorm = 41.2065, GNorm = 2.7114, lr_0 = 8.9209e-04
Validation auc = 0.635211
Epoch 75
Validation auc = 0.622535
Epoch 76
Validation auc = 0.628638
Epoch 77
Validation auc = 0.639437
Epoch 78
Validation auc = 0.660563
Epoch 79
Loss = 5.4777e-01, PNorm = 41.2713, GNorm = 3.4111, lr_0 = 8.8447e-04
Validation auc = 0.665258
Epoch 80
Validation auc = 0.670423
Epoch 81
Validation auc = 0.666667
Epoch 82
Validation auc = 0.655869
Epoch 83
Validation auc = 0.649765
Epoch 84
Loss = 5.1738e-01, PNorm = 41.3113, GNorm = 2.7891, lr_0 = 8.7691e-04
Validation auc = 0.638028
Epoch 85
Validation auc = 0.635211
Epoch 86
Validation auc = 0.636620
Epoch 87
Validation auc = 0.639437
Epoch 88
Validation auc = 0.643662
Epoch 89
Loss = 4.7493e-01, PNorm = 41.3699, GNorm = 2.6341, lr_0 = 8.6941e-04
Validation auc = 0.647887
Epoch 90
Validation auc = 0.648826
Epoch 91
Validation auc = 0.666197
Epoch 92
Validation auc = 0.674648
Epoch 93
Validation auc = 0.677465
Epoch 94
Loss = 5.5329e-01, PNorm = 41.4377, GNorm = 3.8990, lr_0 = 8.6198e-04
Validation auc = 0.681221
Epoch 95
Validation auc = 0.692958
Epoch 96
Validation auc = 0.697183
Epoch 97
Validation auc = 0.686854
Epoch 98
Validation auc = 0.667136
Epoch 99
Loss = 5.0480e-01, PNorm = 41.5209, GNorm = 2.3855, lr_0 = 8.5461e-04
Validation auc = 0.642723
Epoch 100
Validation auc = 0.646948
Epoch 101
Validation auc = 0.646009
Epoch 102
Validation auc = 0.656338
Epoch 103
Validation auc = 0.666667
Epoch 104
Loss = 4.5554e-01, PNorm = 41.5956, GNorm = 1.9447, lr_0 = 8.4730e-04
Validation auc = 0.669014
Epoch 105
Validation auc = 0.661502
Epoch 106
Validation auc = 0.657277
Epoch 107
Validation auc = 0.661972
Epoch 108
Validation auc = 0.671362
Epoch 109
Loss = 4.9870e-01, PNorm = 41.6633, GNorm = 3.1835, lr_0 = 8.4006e-04
Validation auc = 0.678404
Epoch 110
Validation auc = 0.680282
Epoch 111
Validation auc = 0.680751
Epoch 112
Validation auc = 0.681221
Epoch 113
Validation auc = 0.672300
Epoch 114
Loss = 5.2354e-01, PNorm = 41.7399, GNorm = 2.3512, lr_0 = 8.3288e-04
Validation auc = 0.667606
Epoch 115
Validation auc = 0.662911
Epoch 116
Validation auc = 0.654460
Epoch 117
Validation auc = 0.657277
Epoch 118
Validation auc = 0.672300
Epoch 119
Loss = 5.0476e-01, PNorm = 41.8358, GNorm = 3.2639, lr_0 = 8.2576e-04
Validation auc = 0.693897
Epoch 120
Validation auc = 0.697183
Epoch 121
Validation auc = 0.700939
Epoch 122
Validation auc = 0.680282
Epoch 123
Validation auc = 0.678873
Epoch 124
Loss = 4.0270e-01, PNorm = 41.9268, GNorm = 2.1858, lr_0 = 8.1870e-04
Validation auc = 0.679812
Epoch 125
Validation auc = 0.682160
Epoch 126
Validation auc = 0.687793
Epoch 127
Validation auc = 0.697653
Epoch 128
Validation auc = 0.692488
Epoch 129
Loss = 4.5132e-01, PNorm = 42.0280, GNorm = 3.1844, lr_0 = 8.1170e-04
Validation auc = 0.691080
Epoch 130
Validation auc = 0.687793
Epoch 131
Validation auc = 0.692488
Epoch 132
Validation auc = 0.707042
Epoch 133
Validation auc = 0.698592
Epoch 134
Loss = 4.5658e-01, PNorm = 42.1328, GNorm = 3.6524, lr_0 = 8.0476e-04
Validation auc = 0.701878
Epoch 135
Validation auc = 0.692958
Epoch 136
Validation auc = 0.704695
Epoch 137
Validation auc = 0.706103
Epoch 138
Validation auc = 0.707042
Epoch 139
Loss = 4.8805e-01, PNorm = 42.2145, GNorm = 3.3067, lr_0 = 7.9788e-04
Validation auc = 0.709859
Epoch 140
Validation auc = 0.715962
Epoch 141
Validation auc = 0.733333
Epoch 142
Validation auc = 0.737559
Epoch 143
Validation auc = 0.734742
Epoch 144
Loss = 4.4108e-01, PNorm = 42.3014, GNorm = 3.2396, lr_0 = 7.9106e-04
Validation auc = 0.730047
Epoch 145
Validation auc = 0.731455
Epoch 146
Validation auc = 0.723005
Epoch 147
Validation auc = 0.730986
Epoch 148
Validation auc = 0.737559
Epoch 149
Loss = 4.3145e-01, PNorm = 42.4123, GNorm = 3.2729, lr_0 = 7.8430e-04
Validation auc = 0.745070
Epoch 150
Validation auc = 0.763850
Epoch 151
Validation auc = 0.756338
Epoch 152
Validation auc = 0.752113
Epoch 153
Validation auc = 0.743192
Epoch 154
Loss = 4.0937e-01, PNorm = 42.4988, GNorm = 4.3696, lr_0 = 7.7759e-04
Validation auc = 0.743192
Epoch 155
Validation auc = 0.743192
Epoch 156
Validation auc = 0.722535
Epoch 157
Validation auc = 0.707512
Epoch 158
Validation auc = 0.696714
Epoch 159
Loss = 4.1342e-01, PNorm = 42.5807, GNorm = 3.4529, lr_0 = 7.7095e-04
Validation auc = 0.691549
Epoch 160
Validation auc = 0.700469
Epoch 161
Validation auc = 0.710798
Epoch 162
Validation auc = 0.738028
Epoch 163
Validation auc = 0.736150
Epoch 164
Loss = 3.7821e-01, PNorm = 42.6576, GNorm = 2.0713, lr_0 = 7.6436e-04
Validation auc = 0.742723
Epoch 165
Validation auc = 0.738967
Epoch 166
Validation auc = 0.733333
Epoch 167
Validation auc = 0.724413
Epoch 168
Validation auc = 0.722066
Epoch 169
Loss = 3.7799e-01, PNorm = 42.7566, GNorm = 2.0476, lr_0 = 7.5782e-04
Validation auc = 0.722066
Epoch 170
Validation auc = 0.720188
Epoch 171
Validation auc = 0.728638
Epoch 172
Validation auc = 0.729108
Epoch 173
Validation auc = 0.735681
Epoch 174
Loss = 3.8103e-01, PNorm = 42.8446, GNorm = 2.5770, lr_0 = 7.5134e-04
Validation auc = 0.727700
Epoch 175
Validation auc = 0.733333
Epoch 176
Validation auc = 0.730516
Epoch 177
Validation auc = 0.729108
Epoch 178
Validation auc = 0.734742
Epoch 179
Loss = 4.0257e-01, PNorm = 42.9218, GNorm = 2.5655, lr_0 = 7.4492e-04
Validation auc = 0.750235
Epoch 180
Validation auc = 0.741315
Epoch 181
Validation auc = 0.725352
Epoch 182
Validation auc = 0.710798
Epoch 183
Validation auc = 0.714554
Epoch 184
Loss = 3.5298e-01, PNorm = 42.9909, GNorm = 1.9254, lr_0 = 7.3855e-04
Validation auc = 0.726761
Epoch 185
Validation auc = 0.730516
Epoch 186
Validation auc = 0.729577
Epoch 187
Validation auc = 0.715493
Epoch 188
Validation auc = 0.702817
Epoch 189
Loss = 3.3940e-01, PNorm = 43.0687, GNorm = 2.4999, lr_0 = 7.3224e-04
Validation auc = 0.690610
Epoch 190
Validation auc = 0.695305
Epoch 191
Validation auc = 0.707512
Epoch 192
Validation auc = 0.710329
Epoch 193
Validation auc = 0.717371
Epoch 194
Loss = 3.5784e-01, PNorm = 43.1454, GNorm = 4.3917, lr_0 = 7.2598e-04
Validation auc = 0.717371
Epoch 195
Validation auc = 0.718310
Epoch 196
Validation auc = 0.719718
Epoch 197
Validation auc = 0.733333
Epoch 198
Validation auc = 0.730986
Epoch 199
Loss = 3.5423e-01, PNorm = 43.2422, GNorm = 2.2942, lr_0 = 7.1977e-04
Validation auc = 0.715962
Epoch 200
Validation auc = 0.706103
Epoch 201
Validation auc = 0.705164
Epoch 202
Validation auc = 0.716432
Epoch 203
Validation auc = 0.723474
Epoch 204
Loss = 3.6390e-01, PNorm = 43.3271, GNorm = 2.4569, lr_0 = 7.1362e-04
Validation auc = 0.711737
Epoch 205
Validation auc = 0.711268
Epoch 206
Validation auc = 0.715023
Epoch 207
Validation auc = 0.719249
Epoch 208
Validation auc = 0.723474
Epoch 209
Loss = 2.7273e-01, PNorm = 43.4115, GNorm = 3.6572, lr_0 = 7.0752e-04
Validation auc = 0.723474
Epoch 210
Validation auc = 0.719249
Epoch 211
Validation auc = 0.723005
Epoch 212
Validation auc = 0.734272
Epoch 213
Validation auc = 0.731455
Epoch 214
Loss = 3.7233e-01, PNorm = 43.4889, GNorm = 2.5126, lr_0 = 7.0147e-04
Validation auc = 0.721596
Epoch 215
Validation auc = 0.714085
Epoch 216
Validation auc = 0.711737
Epoch 217
Validation auc = 0.716901
Epoch 218
Validation auc = 0.724883
Epoch 219
Loss = 3.0311e-01, PNorm = 43.5604, GNorm = 3.0114, lr_0 = 6.9548e-04
Validation auc = 0.730516
Epoch 220
Validation auc = 0.729577
Epoch 221
Validation auc = 0.733333
Epoch 222
Validation auc = 0.735681
Epoch 223
Validation auc = 0.735681
Epoch 224
Loss = 3.3344e-01, PNorm = 43.6485, GNorm = 2.1899, lr_0 = 6.8953e-04
Validation auc = 0.737559
Epoch 225
Validation auc = 0.743192
Epoch 226
Validation auc = 0.749296
Epoch 227
Validation auc = 0.750235
Epoch 228
Validation auc = 0.754460
Epoch 229
Loss = 3.1298e-01, PNorm = 43.7215, GNorm = 2.8020, lr_0 = 6.8364e-04
Validation auc = 0.744601
Epoch 230
Validation auc = 0.742723
Epoch 231
Validation auc = 0.741315
Epoch 232
Validation auc = 0.740845
Epoch 233
Validation auc = 0.743192
Epoch 234
Loss = 2.9451e-01, PNorm = 43.8068, GNorm = 2.1586, lr_0 = 6.7779e-04
Validation auc = 0.748826
Epoch 235
Validation auc = 0.752582
Epoch 236
Validation auc = 0.754930
Epoch 237
Validation auc = 0.757277
Epoch 238
Validation auc = 0.756338
Epoch 239
Loss = 3.2507e-01, PNorm = 43.8931, GNorm = 2.4427, lr_0 = 6.7200e-04
Validation auc = 0.752582
Epoch 240
Validation auc = 0.741784
Epoch 241
Validation auc = 0.730047
Epoch 242
Validation auc = 0.742723
Epoch 243
Validation auc = 0.737559
Epoch 244
Loss = 2.8611e-01, PNorm = 43.9835, GNorm = 2.9700, lr_0 = 6.6625e-04
Validation auc = 0.728638
Epoch 245
Validation auc = 0.722535
Epoch 246
Validation auc = 0.722535
Epoch 247
Validation auc = 0.729108
Epoch 248
Validation auc = 0.733333
Epoch 249
Loss = 2.2796e-01, PNorm = 44.0774, GNorm = 3.1706, lr_0 = 6.6056e-04
Validation auc = 0.724883
Epoch 250
Validation auc = 0.720188
Epoch 251
Validation auc = 0.730986
Epoch 252
Validation auc = 0.731455
Epoch 253
Validation auc = 0.720188
Epoch 254
Loss = 2.7918e-01, PNorm = 44.1538, GNorm = 2.0900, lr_0 = 6.5491e-04
Validation auc = 0.711268
Epoch 255
Validation auc = 0.721596
Epoch 256
Validation auc = 0.727230
Epoch 257
Validation auc = 0.715023
Epoch 258
Validation auc = 0.705164
Epoch 259
Loss = 3.4441e-01, PNorm = 44.2343, GNorm = 2.7034, lr_0 = 6.4931e-04
Validation auc = 0.710329
Epoch 260
Validation auc = 0.716432
Epoch 261
Validation auc = 0.719718
Epoch 262
Validation auc = 0.726291
Epoch 263
Validation auc = 0.728169
Epoch 264
Loss = 2.9932e-01, PNorm = 44.3186, GNorm = 2.1079, lr_0 = 6.4376e-04
Validation auc = 0.723944
Epoch 265
Validation auc = 0.717371
Epoch 266
Validation auc = 0.722066
Epoch 267
Validation auc = 0.724413
Epoch 268
Validation auc = 0.730047
Epoch 269
Loss = 2.8537e-01, PNorm = 44.3823, GNorm = 2.3619, lr_0 = 6.3826e-04
Validation auc = 0.730516
Epoch 270
Validation auc = 0.735211
Epoch 271
Validation auc = 0.735211
Epoch 272
Validation auc = 0.738498
Epoch 273
Validation auc = 0.738967
Epoch 274
Loss = 2.1924e-01, PNorm = 44.4664, GNorm = 1.8456, lr_0 = 6.3280e-04
Validation auc = 0.737089
Epoch 275
Validation auc = 0.735211
Epoch 276
Validation auc = 0.735681
Epoch 277
Validation auc = 0.740376
Epoch 278
Validation auc = 0.746479
Epoch 279
Loss = 2.8099e-01, PNorm = 44.5584, GNorm = 4.6434, lr_0 = 6.2739e-04
Validation auc = 0.745070
Epoch 280
Validation auc = 0.745070
Epoch 281
Validation auc = 0.739437
Epoch 282
Validation auc = 0.732394
Epoch 283
Validation auc = 0.732394
Epoch 284
Loss = 2.1668e-01, PNorm = 44.6429, GNorm = 2.5599, lr_0 = 6.2203e-04
Validation auc = 0.722535
Epoch 285
Validation auc = 0.714085
Epoch 286
Validation auc = 0.708920
Epoch 287
Validation auc = 0.708920
Epoch 288
Validation auc = 0.710798
Epoch 289
Loss = 2.8424e-01, PNorm = 44.7194, GNorm = 1.3332, lr_0 = 6.1671e-04
Validation auc = 0.712207
Epoch 290
Validation auc = 0.715493
Epoch 291
Validation auc = 0.721127
Epoch 292
Validation auc = 0.728638
Epoch 293
Validation auc = 0.733803
Epoch 294
Loss = 2.8917e-01, PNorm = 44.7856, GNorm = 3.2349, lr_0 = 6.1144e-04
Validation auc = 0.733803
Epoch 295
Validation auc = 0.738028
Epoch 296
Validation auc = 0.739437
Epoch 297
Validation auc = 0.739437
Epoch 298
Validation auc = 0.748826
Epoch 299
Loss = 2.6445e-01, PNorm = 44.8719, GNorm = 2.5701, lr_0 = 6.0621e-04
Validation auc = 0.759155
Model 0 best validation auc = 0.763850 on epoch 150
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.913305
Ensemble test auc = 0.913305
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.523944
Epoch 1
Validation auc = 0.680751
Epoch 2
Validation auc = 0.669014
Epoch 3
Validation auc = 0.669484
Epoch 4
Loss = 1.3235e+00, PNorm = 40.6679, GNorm = 10.8348, lr_0 = 6.5000e-04
Validation auc = 0.676056
Epoch 5
Validation auc = 0.673709
Epoch 6
Validation auc = 0.669014
Epoch 7
Validation auc = 0.662441
Epoch 8
Validation auc = 0.657277
Epoch 9
Loss = 9.6073e-01, PNorm = 40.7089, GNorm = 10.5573, lr_0 = 9.9743e-04
Validation auc = 0.638967
Epoch 10
Validation auc = 0.656338
Epoch 11
Validation auc = 0.657277
Epoch 12
Validation auc = 0.656808
Epoch 13
Validation auc = 0.662911
Epoch 14
Loss = 7.9041e-01, PNorm = 40.7595, GNorm = 8.7374, lr_0 = 9.8890e-04
Validation auc = 0.654460
Epoch 15
Validation auc = 0.696714
Epoch 16
Validation auc = 0.658685
Epoch 17
Validation auc = 0.676526
Epoch 18
Validation auc = 0.673239
Epoch 19
Loss = 8.1752e-01, PNorm = 40.8060, GNorm = 8.7018, lr_0 = 9.8045e-04
Validation auc = 0.673239
Epoch 20
Validation auc = 0.690610
Epoch 21
Validation auc = 0.698122
Epoch 22
Validation auc = 0.694836
Epoch 23
Validation auc = 0.692488
Epoch 24
Loss = 5.8658e-01, PNorm = 40.8512, GNorm = 4.6600, lr_0 = 9.7207e-04
Validation auc = 0.696244
Epoch 25
Validation auc = 0.693897
Epoch 26
Validation auc = 0.702817
Epoch 27
Validation auc = 0.697653
Epoch 28
Validation auc = 0.662911
Epoch 29
Loss = 6.6766e-01, PNorm = 40.8789, GNorm = 6.1643, lr_0 = 9.6376e-04
Validation auc = 0.657277
Epoch 30
Validation auc = 0.660094
Epoch 31
Validation auc = 0.653521
Epoch 32
Validation auc = 0.643192
Epoch 33
Validation auc = 0.643662
Epoch 34
Loss = 6.8848e-01, PNorm = 40.9078, GNorm = 5.5601, lr_0 = 9.5552e-04
Validation auc = 0.646948
Epoch 35
Validation auc = 0.674648
Epoch 36
Validation auc = 0.667136
Epoch 37
Validation auc = 0.666667
Epoch 38
Validation auc = 0.664789
Epoch 39
Loss = 5.7763e-01, PNorm = 40.9449, GNorm = 3.8607, lr_0 = 9.4735e-04
Validation auc = 0.680751
Epoch 40
Validation auc = 0.672300
Epoch 41
Validation auc = 0.664319
Epoch 42
Validation auc = 0.663380
Epoch 43
Validation auc = 0.671831
Epoch 44
Loss = 4.8183e-01, PNorm = 40.9914, GNorm = 3.5697, lr_0 = 9.3925e-04
Validation auc = 0.708920
Epoch 45
Validation auc = 0.690141
Epoch 46
Validation auc = 0.697653
Epoch 47
Validation auc = 0.684038
Epoch 48
Validation auc = 0.668545
Epoch 49
Loss = 6.0177e-01, PNorm = 41.0231, GNorm = 3.2645, lr_0 = 9.3122e-04
Validation auc = 0.666667
Epoch 50
Validation auc = 0.664319
Epoch 51
Validation auc = 0.668075
Epoch 52
Validation auc = 0.665258
Epoch 53
Validation auc = 0.665728
Epoch 54
Loss = 5.6454e-01, PNorm = 41.0591, GNorm = 2.8316, lr_0 = 9.2326e-04
Validation auc = 0.663380
Epoch 55
Validation auc = 0.657746
Epoch 56
Validation auc = 0.651643
Epoch 57
Validation auc = 0.662911
Epoch 58
Validation auc = 0.661972
Epoch 59
Loss = 5.8134e-01, PNorm = 41.1075, GNorm = 3.9781, lr_0 = 9.1537e-04
Validation auc = 0.656808
Epoch 60
Validation auc = 0.647418
Epoch 61
Validation auc = 0.644131
Epoch 62
Validation auc = 0.649765
Epoch 63
Validation auc = 0.653052
Epoch 64
Loss = 5.2201e-01, PNorm = 41.1730, GNorm = 3.2161, lr_0 = 9.0754e-04
Validation auc = 0.653521
Epoch 65
Validation auc = 0.656808
Epoch 66
Validation auc = 0.655869
Epoch 67
Validation auc = 0.651643
Epoch 68
Validation auc = 0.646948
Epoch 69
Loss = 5.9220e-01, PNorm = 41.2163, GNorm = 3.6572, lr_0 = 8.9978e-04
Validation auc = 0.649296
Epoch 70
Validation auc = 0.638967
Epoch 71
Validation auc = 0.629108
Epoch 72
Validation auc = 0.632864
Epoch 73
Validation auc = 0.638028
Epoch 74
Loss = 5.5007e-01, PNorm = 41.2861, GNorm = 3.2879, lr_0 = 8.9209e-04
Validation auc = 0.640845
Epoch 75
Validation auc = 0.642254
Epoch 76
Validation auc = 0.640845
Epoch 77
Validation auc = 0.630047
Epoch 78
Validation auc = 0.626291
Epoch 79
Loss = 5.0204e-01, PNorm = 41.3389, GNorm = 3.6914, lr_0 = 8.8447e-04
Validation auc = 0.623474
Epoch 80
Validation auc = 0.619718
Epoch 81
Validation auc = 0.623474
Epoch 82
Validation auc = 0.629577
Epoch 83
Validation auc = 0.636620
Epoch 84
Loss = 4.2371e-01, PNorm = 41.4078, GNorm = 3.2763, lr_0 = 8.7691e-04
Validation auc = 0.642254
Epoch 85
Validation auc = 0.640376
Epoch 86
Validation auc = 0.649296
Epoch 87
Validation auc = 0.655399
Epoch 88
Validation auc = 0.659155
Epoch 89
Loss = 4.7340e-01, PNorm = 41.4651, GNorm = 2.1608, lr_0 = 8.6941e-04
Validation auc = 0.651643
Epoch 90
Validation auc = 0.646009
Epoch 91
Validation auc = 0.656338
Epoch 92
Validation auc = 0.660094
Epoch 93
Validation auc = 0.662911
Epoch 94
Loss = 4.8955e-01, PNorm = 41.5409, GNorm = 2.3912, lr_0 = 8.6198e-04
Validation auc = 0.644131
Epoch 95
Validation auc = 0.646009
Epoch 96
Validation auc = 0.643662
Epoch 97
Validation auc = 0.638498
Epoch 98
Validation auc = 0.638498
Epoch 99
Loss = 4.7300e-01, PNorm = 41.5982, GNorm = 2.8803, lr_0 = 8.5461e-04
Validation auc = 0.634742
Epoch 100
Validation auc = 0.650235
Epoch 101
Validation auc = 0.665258
Epoch 102
Validation auc = 0.668075
Epoch 103
Validation auc = 0.662911
Epoch 104
Loss = 4.9217e-01, PNorm = 41.6819, GNorm = 2.8168, lr_0 = 8.4730e-04
Validation auc = 0.661033
Epoch 105
Validation auc = 0.657277
Epoch 106
Validation auc = 0.643662
Epoch 107
Validation auc = 0.639437
Epoch 108
Validation auc = 0.644601
Epoch 109
Loss = 4.1850e-01, PNorm = 41.7752, GNorm = 2.7337, lr_0 = 8.4006e-04
Validation auc = 0.648357
Epoch 110
Validation auc = 0.657746
Epoch 111
Validation auc = 0.658685
Epoch 112
Validation auc = 0.659155
Epoch 113
Validation auc = 0.652582
Epoch 114
Loss = 4.3138e-01, PNorm = 41.8783, GNorm = 2.6740, lr_0 = 8.3288e-04
Validation auc = 0.652113
Epoch 115
Validation auc = 0.652113
Epoch 116
Validation auc = 0.651643
Epoch 117
Validation auc = 0.653521
Epoch 118
Validation auc = 0.662441
Epoch 119
Loss = 4.1232e-01, PNorm = 41.9849, GNorm = 3.0759, lr_0 = 8.2576e-04
Validation auc = 0.668545
Epoch 120
Validation auc = 0.676526
Epoch 121
Validation auc = 0.668075
Epoch 122
Validation auc = 0.670423
Epoch 123
Validation auc = 0.675117
Epoch 124
Loss = 4.1703e-01, PNorm = 42.0741, GNorm = 2.3914, lr_0 = 8.1870e-04
Validation auc = 0.678404
Epoch 125
Validation auc = 0.675587
Epoch 126
Validation auc = 0.673709
Epoch 127
Validation auc = 0.673709
Epoch 128
Validation auc = 0.684038
Epoch 129
Loss = 4.0811e-01, PNorm = 42.1763, GNorm = 2.8827, lr_0 = 8.1170e-04
Validation auc = 0.680282
Epoch 130
Validation auc = 0.679343
Epoch 131
Validation auc = 0.688263
Epoch 132
Validation auc = 0.690141
Epoch 133
Validation auc = 0.687793
Epoch 134
Loss = 4.5039e-01, PNorm = 42.2607, GNorm = 3.5740, lr_0 = 8.0476e-04
Validation auc = 0.692488
Epoch 135
Validation auc = 0.695775
Epoch 136
Validation auc = 0.694836
Epoch 137
Validation auc = 0.700469
Epoch 138
Validation auc = 0.700939
Epoch 139
Loss = 4.2703e-01, PNorm = 42.3466, GNorm = 2.6820, lr_0 = 7.9788e-04
Validation auc = 0.698592
Epoch 140
Validation auc = 0.694836
Epoch 141
Validation auc = 0.703286
Epoch 142
Validation auc = 0.698592
Epoch 143
Validation auc = 0.694836
Epoch 144
Loss = 4.6522e-01, PNorm = 42.4414, GNorm = 2.5665, lr_0 = 7.9106e-04
Validation auc = 0.699061
Epoch 145
Validation auc = 0.708920
Epoch 146
Validation auc = 0.707512
Epoch 147
Validation auc = 0.712207
Epoch 148
Validation auc = 0.714085
Epoch 149
Loss = 3.5129e-01, PNorm = 42.5515, GNorm = 2.9054, lr_0 = 7.8430e-04
Validation auc = 0.712676
Epoch 150
Validation auc = 0.711737
Epoch 151
Validation auc = 0.712207
Epoch 152
Validation auc = 0.721127
Epoch 153
Validation auc = 0.717840
Epoch 154
Loss = 3.5110e-01, PNorm = 42.6587, GNorm = 2.8926, lr_0 = 7.7759e-04
Validation auc = 0.715023
Epoch 155
Validation auc = 0.715493
Epoch 156
Validation auc = 0.723474
Epoch 157
Validation auc = 0.729577
Epoch 158
Validation auc = 0.725352
Epoch 159
Loss = 3.2225e-01, PNorm = 42.7444, GNorm = 2.4868, lr_0 = 7.7095e-04
Validation auc = 0.713615
Epoch 160
Validation auc = 0.708920
Epoch 161
Validation auc = 0.717840
Epoch 162
Validation auc = 0.719249
Epoch 163
Validation auc = 0.711737
Epoch 164
Loss = 5.0058e-01, PNorm = 42.8353, GNorm = 3.8402, lr_0 = 7.6436e-04
Validation auc = 0.707981
Epoch 165
Validation auc = 0.712207
Epoch 166
Validation auc = 0.713615
Epoch 167
Validation auc = 0.706573
Epoch 168
Validation auc = 0.710329
Epoch 169
Loss = 4.2041e-01, PNorm = 42.9160, GNorm = 2.5037, lr_0 = 7.5782e-04
Validation auc = 0.716432
Epoch 170
Validation auc = 0.716901
Epoch 171
Validation auc = 0.723944
Epoch 172
Validation auc = 0.735681
Epoch 173
Validation auc = 0.745540
Epoch 174
Loss = 3.7577e-01, PNorm = 43.0363, GNorm = 3.1675, lr_0 = 7.5134e-04
Validation auc = 0.748357
Epoch 175
Validation auc = 0.750704
Epoch 176
Validation auc = 0.739906
Epoch 177
Validation auc = 0.728638
Epoch 178
Validation auc = 0.730516
Epoch 179
Loss = 3.2812e-01, PNorm = 43.1468, GNorm = 2.8677, lr_0 = 7.4492e-04
Validation auc = 0.727700
Epoch 180
Validation auc = 0.745540
Epoch 181
Validation auc = 0.744601
Epoch 182
Validation auc = 0.746948
Epoch 183
Validation auc = 0.746479
Epoch 184
Loss = 3.6043e-01, PNorm = 43.2585, GNorm = 3.0121, lr_0 = 7.3855e-04
Validation auc = 0.738028
Epoch 185
Validation auc = 0.729108
Epoch 186
Validation auc = 0.725822
Epoch 187
Validation auc = 0.722066
Epoch 188
Validation auc = 0.714554
Epoch 189
Loss = 3.7800e-01, PNorm = 43.3535, GNorm = 2.7378, lr_0 = 7.3224e-04
Validation auc = 0.718779
Epoch 190
Validation auc = 0.722066
Epoch 191
Validation auc = 0.724883
Epoch 192
Validation auc = 0.722535
Epoch 193
Validation auc = 0.720657
Epoch 194
Loss = 3.5818e-01, PNorm = 43.4567, GNorm = 3.2224, lr_0 = 7.2598e-04
Validation auc = 0.717840
Epoch 195
Validation auc = 0.717371
Epoch 196
Validation auc = 0.718779
Epoch 197
Validation auc = 0.715962
Epoch 198
Validation auc = 0.717840
Epoch 199
Loss = 3.4056e-01, PNorm = 43.5553, GNorm = 2.2916, lr_0 = 7.1977e-04
Validation auc = 0.725822
Epoch 200
Validation auc = 0.733333
Epoch 201
Validation auc = 0.723944
Epoch 202
Validation auc = 0.723005
Epoch 203
Validation auc = 0.722535
Epoch 204
Loss = 3.3288e-01, PNorm = 43.6507, GNorm = 2.0318, lr_0 = 7.1362e-04
Validation auc = 0.722066
Epoch 205
Validation auc = 0.722535
Epoch 206
Validation auc = 0.725822
Epoch 207
Validation auc = 0.730986
Epoch 208
Validation auc = 0.740845
Epoch 209
Loss = 3.3900e-01, PNorm = 43.7688, GNorm = 2.3367, lr_0 = 7.0752e-04
Validation auc = 0.732864
Epoch 210
Validation auc = 0.723944
Epoch 211
Validation auc = 0.726291
Epoch 212
Validation auc = 0.730516
Epoch 213
Validation auc = 0.728169
Epoch 214
Loss = 3.6302e-01, PNorm = 43.8446, GNorm = 2.9859, lr_0 = 7.0147e-04
Validation auc = 0.748357
Epoch 215
Validation auc = 0.751643
Epoch 216
Validation auc = 0.755869
Epoch 217
Validation auc = 0.755399
Epoch 218
Validation auc = 0.749765
Epoch 219
Loss = 3.5437e-01, PNorm = 43.9305, GNorm = 2.6024, lr_0 = 6.9548e-04
Validation auc = 0.749765
Epoch 220
Validation auc = 0.745540
Epoch 221
Validation auc = 0.746948
Epoch 222
Validation auc = 0.747887
Epoch 223
Validation auc = 0.747418
Epoch 224
Loss = 2.8031e-01, PNorm = 44.0147, GNorm = 2.4544, lr_0 = 6.8953e-04
Validation auc = 0.749765
Epoch 225
Validation auc = 0.750235
Epoch 226
Validation auc = 0.757277
Epoch 227
Validation auc = 0.760094
Epoch 228
Validation auc = 0.768545
Epoch 229
Loss = 3.6740e-01, PNorm = 44.1040, GNorm = 2.4796, lr_0 = 6.8364e-04
Validation auc = 0.769484
Epoch 230
Validation auc = 0.770892
Epoch 231
Validation auc = 0.773709
Epoch 232
Validation auc = 0.765258
Epoch 233
Validation auc = 0.761972
Epoch 234
Loss = 2.5582e-01, PNorm = 44.1900, GNorm = 2.6549, lr_0 = 6.7779e-04
Validation auc = 0.756338
Epoch 235
Validation auc = 0.754930
Epoch 236
Validation auc = 0.752582
Epoch 237
Validation auc = 0.761033
Epoch 238
Validation auc = 0.761033
Epoch 239
Loss = 2.8032e-01, PNorm = 44.2854, GNorm = 2.3862, lr_0 = 6.7200e-04
Validation auc = 0.762441
Epoch 240
Validation auc = 0.761972
Epoch 241
Validation auc = 0.765258
Epoch 242
Validation auc = 0.756338
Epoch 243
Validation auc = 0.750704
Epoch 244
Loss = 2.8897e-01, PNorm = 44.3639, GNorm = 3.0296, lr_0 = 6.6625e-04
Validation auc = 0.737089
Epoch 245
Validation auc = 0.742723
Epoch 246
Validation auc = 0.746948
Epoch 247
Validation auc = 0.752113
Epoch 248
Validation auc = 0.749765
Epoch 249
Loss = 3.1562e-01, PNorm = 44.4432, GNorm = 1.9728, lr_0 = 6.6056e-04
Validation auc = 0.749765
Epoch 250
Validation auc = 0.753052
Epoch 251
Validation auc = 0.753052
Epoch 252
Validation auc = 0.746009
Epoch 253
Validation auc = 0.741784
Epoch 254
Loss = 3.7903e-01, PNorm = 44.5402, GNorm = 2.7568, lr_0 = 6.5491e-04
Validation auc = 0.737559
Epoch 255
Validation auc = 0.733333
Epoch 256
Validation auc = 0.734272
Epoch 257
Validation auc = 0.739906
Epoch 258
Validation auc = 0.749296
Epoch 259
Loss = 3.3796e-01, PNorm = 44.6455, GNorm = 1.8949, lr_0 = 6.4931e-04
Validation auc = 0.758685
Epoch 260
Validation auc = 0.754460
Epoch 261
Validation auc = 0.754460
Epoch 262
Validation auc = 0.750235
Epoch 263
Validation auc = 0.735211
Epoch 264
Loss = 2.6654e-01, PNorm = 44.7238, GNorm = 3.9596, lr_0 = 6.4376e-04
Validation auc = 0.725352
Epoch 265
Validation auc = 0.720188
Epoch 266
Validation auc = 0.719249
Epoch 267
Validation auc = 0.720657
Epoch 268
Validation auc = 0.725822
Epoch 269
Loss = 2.6060e-01, PNorm = 44.7848, GNorm = 1.9669, lr_0 = 6.3826e-04
Validation auc = 0.727230
Epoch 270
Validation auc = 0.728638
Epoch 271
Validation auc = 0.727700
Epoch 272
Validation auc = 0.730986
Epoch 273
Validation auc = 0.736620
Epoch 274
Loss = 2.0992e-01, PNorm = 44.8446, GNorm = 1.6835, lr_0 = 6.3280e-04
Validation auc = 0.746479
Epoch 275
Validation auc = 0.751643
Epoch 276
Validation auc = 0.752113
Epoch 277
Validation auc = 0.751174
Epoch 278
Validation auc = 0.753052
Epoch 279
Loss = 3.3659e-01, PNorm = 44.9187, GNorm = 1.9939, lr_0 = 6.2739e-04
Validation auc = 0.747418
Epoch 280
Validation auc = 0.742723
Epoch 281
Validation auc = 0.739437
Epoch 282
Validation auc = 0.737089
Epoch 283
Validation auc = 0.732394
Epoch 284
Loss = 3.1821e-01, PNorm = 44.9958, GNorm = 3.0341, lr_0 = 6.2203e-04
Validation auc = 0.727700
Epoch 285
Validation auc = 0.733333
Epoch 286
Validation auc = 0.726761
Epoch 287
Validation auc = 0.729108
Epoch 288
Validation auc = 0.731925
Epoch 289
Loss = 3.7322e-01, PNorm = 45.0684, GNorm = 2.0801, lr_0 = 6.1671e-04
Validation auc = 0.734742
Epoch 290
Validation auc = 0.736620
Epoch 291
Validation auc = 0.733333
Epoch 292
Validation auc = 0.731455
Epoch 293
Validation auc = 0.733803
Epoch 294
Loss = 2.2038e-01, PNorm = 45.1355, GNorm = 2.4326, lr_0 = 6.1144e-04
Validation auc = 0.736620
Epoch 295
Validation auc = 0.735681
Epoch 296
Validation auc = 0.732394
Epoch 297
Validation auc = 0.731925
Epoch 298
Validation auc = 0.733333
Epoch 299
Loss = 1.8042e-01, PNorm = 45.2102, GNorm = 2.2650, lr_0 = 6.0621e-04
Validation auc = 0.730047
Model 0 best validation auc = 0.773709 on epoch 231
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.927152
Ensemble test auc = 0.927152
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.564319
Epoch 1
Validation auc = 0.687793
Epoch 2
Validation auc = 0.679343
Epoch 3
Validation auc = 0.688263
Epoch 4
Loss = 1.4051e+00, PNorm = 40.6672, GNorm = 11.7362, lr_0 = 6.5000e-04
Validation auc = 0.707042
Epoch 5
Validation auc = 0.727230
Epoch 6
Validation auc = 0.702817
Epoch 7
Validation auc = 0.699531
Epoch 8
Validation auc = 0.690610
Epoch 9
Loss = 9.0930e-01, PNorm = 40.7167, GNorm = 8.7234, lr_0 = 9.9743e-04
Validation auc = 0.692958
Epoch 10
Validation auc = 0.653991
Epoch 11
Validation auc = 0.620188
Epoch 12
Validation auc = 0.623005
Epoch 13
Validation auc = 0.619718
Epoch 14
Loss = 8.4163e-01, PNorm = 40.7653, GNorm = 8.5683, lr_0 = 9.8890e-04
Validation auc = 0.683568
Epoch 15
Validation auc = 0.682160
Epoch 16
Validation auc = 0.720657
Epoch 17
Validation auc = 0.682629
Epoch 18
Validation auc = 0.700939
Epoch 19
Loss = 6.5092e-01, PNorm = 40.8110, GNorm = 4.1391, lr_0 = 9.8045e-04
Validation auc = 0.750235
Epoch 20
Validation auc = 0.746009
Epoch 21
Validation auc = 0.715493
Epoch 22
Validation auc = 0.694366
Epoch 23
Validation auc = 0.694836
Epoch 24
Loss = 7.0423e-01, PNorm = 40.8549, GNorm = 5.6547, lr_0 = 9.7207e-04
Validation auc = 0.710329
Epoch 25
Validation auc = 0.689671
Epoch 26
Validation auc = 0.665728
Epoch 27
Validation auc = 0.649765
Epoch 28
Validation auc = 0.646948
Epoch 29
Loss = 6.3085e-01, PNorm = 40.8841, GNorm = 4.9898, lr_0 = 9.6376e-04
Validation auc = 0.654460
Epoch 30
Validation auc = 0.662441
Epoch 31
Validation auc = 0.669953
Epoch 32
Validation auc = 0.658216
Epoch 33
Validation auc = 0.670423
Epoch 34
Loss = 6.9486e-01, PNorm = 40.9205, GNorm = 5.9975, lr_0 = 9.5552e-04
Validation auc = 0.672770
Epoch 35
Validation auc = 0.678404
Epoch 36
Validation auc = 0.697653
Epoch 37
Validation auc = 0.718779
Epoch 38
Validation auc = 0.688732
Epoch 39
Loss = 6.3885e-01, PNorm = 40.9647, GNorm = 4.9264, lr_0 = 9.4735e-04
Validation auc = 0.689202
Epoch 40
Validation auc = 0.726761
Epoch 41
Validation auc = 0.745070
Epoch 42
Validation auc = 0.736150
Epoch 43
Validation auc = 0.708920
Epoch 44
Loss = 4.8536e-01, PNorm = 41.0070, GNorm = 3.8567, lr_0 = 9.3925e-04
Validation auc = 0.700469
Epoch 45
Validation auc = 0.673709
Epoch 46
Validation auc = 0.660094
Epoch 47
Validation auc = 0.650704
Epoch 48
Validation auc = 0.646009
Epoch 49
Loss = 5.8854e-01, PNorm = 41.0510, GNorm = 5.6494, lr_0 = 9.3122e-04
Validation auc = 0.644131
Epoch 50
Validation auc = 0.646948
Epoch 51
Validation auc = 0.649765
Epoch 52
Validation auc = 0.635681
Epoch 53
Validation auc = 0.637559
Epoch 54
Loss = 6.0475e-01, PNorm = 41.1008, GNorm = 3.8984, lr_0 = 9.2326e-04
Validation auc = 0.648826
Epoch 55
Validation auc = 0.648826
Epoch 56
Validation auc = 0.646948
Epoch 57
Validation auc = 0.645540
Epoch 58
Validation auc = 0.647418
Epoch 59
Loss = 6.2247e-01, PNorm = 41.1582, GNorm = 4.4028, lr_0 = 9.1537e-04
Validation auc = 0.665258
Epoch 60
Validation auc = 0.674648
Epoch 61
Validation auc = 0.669953
Epoch 62
Validation auc = 0.664319
Epoch 63
Validation auc = 0.662441
Epoch 64
Loss = 5.2638e-01, PNorm = 41.1913, GNorm = 3.4416, lr_0 = 9.0754e-04
Validation auc = 0.669014
Epoch 65
Validation auc = 0.681221
Epoch 66
Validation auc = 0.665728
Epoch 67
Validation auc = 0.667606
Epoch 68
Validation auc = 0.669953
Epoch 69
Loss = 5.2159e-01, PNorm = 41.2450, GNorm = 3.2208, lr_0 = 8.9978e-04
Validation auc = 0.670892
Epoch 70
Validation auc = 0.668545
Epoch 71
Validation auc = 0.693897
Epoch 72
Validation auc = 0.674178
Epoch 73
Validation auc = 0.659155
Epoch 74
Loss = 5.5974e-01, PNorm = 41.2845, GNorm = 2.7567, lr_0 = 8.9209e-04
Validation auc = 0.647418
Epoch 75
Validation auc = 0.632394
Epoch 76
Validation auc = 0.635681
Epoch 77
Validation auc = 0.644131
Epoch 78
Validation auc = 0.653991
Epoch 79
Loss = 5.5882e-01, PNorm = 41.3513, GNorm = 3.4976, lr_0 = 8.8447e-04
Validation auc = 0.656808
Epoch 80
Validation auc = 0.654460
Epoch 81
Validation auc = 0.657277
Epoch 82
Validation auc = 0.656338
Epoch 83
Validation auc = 0.656808
Epoch 84
Loss = 4.9009e-01, PNorm = 41.4135, GNorm = 3.0505, lr_0 = 8.7691e-04
Validation auc = 0.653521
Epoch 85
Validation auc = 0.651174
Epoch 86
Validation auc = 0.656808
Epoch 87
Validation auc = 0.669953
Epoch 88
Validation auc = 0.681221
Epoch 89
Loss = 5.8163e-01, PNorm = 41.4696, GNorm = 2.6076, lr_0 = 8.6941e-04
Validation auc = 0.671362
Epoch 90
Validation auc = 0.671362
Epoch 91
Validation auc = 0.673709
Epoch 92
Validation auc = 0.667606
Epoch 93
Validation auc = 0.662441
Epoch 94
Loss = 6.2416e-01, PNorm = 41.5626, GNorm = 3.7212, lr_0 = 8.6198e-04
Validation auc = 0.657746
Epoch 95
Validation auc = 0.655399
Epoch 96
Validation auc = 0.650235
Epoch 97
Validation auc = 0.646479
Epoch 98
Validation auc = 0.650704
Epoch 99
Loss = 5.6097e-01, PNorm = 41.6217, GNorm = 3.2718, lr_0 = 8.5461e-04
Validation auc = 0.662441
Epoch 100
Validation auc = 0.678404
Epoch 101
Validation auc = 0.686854
Epoch 102
Validation auc = 0.705164
Epoch 103
Validation auc = 0.707512
Epoch 104
Loss = 4.3506e-01, PNorm = 41.7065, GNorm = 2.9852, lr_0 = 8.4730e-04
Validation auc = 0.709859
Epoch 105
Validation auc = 0.709859
Epoch 106
Validation auc = 0.712676
Epoch 107
Validation auc = 0.710798
Epoch 108
Validation auc = 0.722066
Epoch 109
Loss = 4.6150e-01, PNorm = 41.7788, GNorm = 2.9168, lr_0 = 8.4006e-04
Validation auc = 0.724413
Epoch 110
Validation auc = 0.724883
Epoch 111
Validation auc = 0.725822
Epoch 112
Validation auc = 0.717840
Epoch 113
Validation auc = 0.714085
Epoch 114
Loss = 5.3994e-01, PNorm = 41.8744, GNorm = 3.8980, lr_0 = 8.3288e-04
Validation auc = 0.714085
Epoch 115
Validation auc = 0.705634
Epoch 116
Validation auc = 0.713615
Epoch 117
Validation auc = 0.736620
Epoch 118
Validation auc = 0.756808
Epoch 119
Loss = 5.4242e-01, PNorm = 41.9537, GNorm = 2.9311, lr_0 = 8.2576e-04
Validation auc = 0.768545
Epoch 120
Validation auc = 0.764789
Epoch 121
Validation auc = 0.753521
Epoch 122
Validation auc = 0.745070
Epoch 123
Validation auc = 0.739437
Epoch 124
Loss = 4.1503e-01, PNorm = 42.0662, GNorm = 2.6455, lr_0 = 8.1870e-04
Validation auc = 0.732864
Epoch 125
Validation auc = 0.730516
Epoch 126
Validation auc = 0.731455
Epoch 127
Validation auc = 0.727700
Epoch 128
Validation auc = 0.730986
Epoch 129
Loss = 3.8873e-01, PNorm = 42.1833, GNorm = 2.6015, lr_0 = 8.1170e-04
Validation auc = 0.727700
Epoch 130
Validation auc = 0.708920
Epoch 131
Validation auc = 0.706103
Epoch 132
Validation auc = 0.703756
Epoch 133
Validation auc = 0.705634
Epoch 134
Loss = 3.4970e-01, PNorm = 42.2865, GNorm = 2.1979, lr_0 = 8.0476e-04
Validation auc = 0.707512
Epoch 135
Validation auc = 0.707512
Epoch 136
Validation auc = 0.699061
Epoch 137
Validation auc = 0.696244
Epoch 138
Validation auc = 0.694836
Epoch 139
Loss = 3.5926e-01, PNorm = 42.3642, GNorm = 2.6086, lr_0 = 7.9788e-04
Validation auc = 0.695775
Epoch 140
Validation auc = 0.700469
Epoch 141
Validation auc = 0.702817
Epoch 142
Validation auc = 0.707981
Epoch 143
Validation auc = 0.708920
Epoch 144
Loss = 3.5823e-01, PNorm = 42.4626, GNorm = 2.9428, lr_0 = 7.9106e-04
Validation auc = 0.715023
Epoch 145
Validation auc = 0.728169
Epoch 146
Validation auc = 0.735681
Epoch 147
Validation auc = 0.730047
Epoch 148
Validation auc = 0.735681
Epoch 149
Loss = 4.0792e-01, PNorm = 42.5467, GNorm = 3.5513, lr_0 = 7.8430e-04
Validation auc = 0.728169
Epoch 150
Validation auc = 0.714085
Epoch 151
Validation auc = 0.711268
Epoch 152
Validation auc = 0.700469
Epoch 153
Validation auc = 0.688263
Epoch 154
Loss = 4.4545e-01, PNorm = 42.6251, GNorm = 3.1155, lr_0 = 7.7759e-04
Validation auc = 0.681221
Epoch 155
Validation auc = 0.681221
Epoch 156
Validation auc = 0.676526
Epoch 157
Validation auc = 0.685446
Epoch 158
Validation auc = 0.697653
Epoch 159
Loss = 3.7058e-01, PNorm = 42.6939, GNorm = 2.3437, lr_0 = 7.7095e-04
Validation auc = 0.704695
Epoch 160
Validation auc = 0.710798
Epoch 161
Validation auc = 0.711268
Epoch 162
Validation auc = 0.709859
Epoch 163
Validation auc = 0.706573
Epoch 164
Loss = 3.3321e-01, PNorm = 42.7721, GNorm = 3.6732, lr_0 = 7.6436e-04
Validation auc = 0.709390
Epoch 165
Validation auc = 0.709390
Epoch 166
Validation auc = 0.707512
Epoch 167
Validation auc = 0.710329
Epoch 168
Validation auc = 0.718779
Epoch 169
Loss = 3.2255e-01, PNorm = 42.8506, GNorm = 2.5339, lr_0 = 7.5782e-04
Validation auc = 0.730516
Epoch 170
Validation auc = 0.737559
Epoch 171
Validation auc = 0.741315
Epoch 172
Validation auc = 0.743662
Epoch 173
Validation auc = 0.738498
Epoch 174
Loss = 3.3780e-01, PNorm = 42.9225, GNorm = 2.9563, lr_0 = 7.5134e-04
Validation auc = 0.731455
Epoch 175
Validation auc = 0.728638
Epoch 176
Validation auc = 0.728638
Epoch 177
Validation auc = 0.725352
Epoch 178
Validation auc = 0.732864
Epoch 179
Loss = 3.8685e-01, PNorm = 43.0155, GNorm = 2.7695, lr_0 = 7.4492e-04
Validation auc = 0.732394
Epoch 180
Validation auc = 0.735681
Epoch 181
Validation auc = 0.738498
Epoch 182
Validation auc = 0.745070
Epoch 183
Validation auc = 0.743662
Epoch 184
Loss = 3.6530e-01, PNorm = 43.1054, GNorm = 2.9411, lr_0 = 7.3855e-04
Validation auc = 0.738498
Epoch 185
Validation auc = 0.733803
Epoch 186
Validation auc = 0.729577
Epoch 187
Validation auc = 0.732394
Epoch 188
Validation auc = 0.731925
Epoch 189
Loss = 3.4936e-01, PNorm = 43.1918, GNorm = 2.5728, lr_0 = 7.3224e-04
Validation auc = 0.730986
Epoch 190
Validation auc = 0.731925
Epoch 191
Validation auc = 0.731925
Epoch 192
Validation auc = 0.731925
Epoch 193
Validation auc = 0.733803
Epoch 194
Loss = 3.4340e-01, PNorm = 43.2851, GNorm = 2.7547, lr_0 = 7.2598e-04
Validation auc = 0.735211
Epoch 195
Validation auc = 0.745540
Epoch 196
Validation auc = 0.744601
Epoch 197
Validation auc = 0.747887
Epoch 198
Validation auc = 0.741315
Epoch 199
Loss = 3.3705e-01, PNorm = 43.3819, GNorm = 2.7729, lr_0 = 7.1977e-04
Validation auc = 0.746009
Epoch 200
Validation auc = 0.744601
Epoch 201
Validation auc = 0.738028
Epoch 202
Validation auc = 0.735681
Epoch 203
Validation auc = 0.731455
Epoch 204
Loss = 2.8032e-01, PNorm = 43.4714, GNorm = 2.1922, lr_0 = 7.1362e-04
Validation auc = 0.727700
Epoch 205
Validation auc = 0.724883
Epoch 206
Validation auc = 0.727230
Epoch 207
Validation auc = 0.727230
Epoch 208
Validation auc = 0.725822
Epoch 209
Loss = 3.2756e-01, PNorm = 43.5704, GNorm = 1.8834, lr_0 = 7.0752e-04
Validation auc = 0.726761
Epoch 210
Validation auc = 0.722066
Epoch 211
Validation auc = 0.723474
Epoch 212
Validation auc = 0.717371
Epoch 213
Validation auc = 0.718779
Epoch 214
Loss = 3.5166e-01, PNorm = 43.6739, GNorm = 3.3102, lr_0 = 7.0147e-04
Validation auc = 0.723474
Epoch 215
Validation auc = 0.727230
Epoch 216
Validation auc = 0.728638
Epoch 217
Validation auc = 0.728638
Epoch 218
Validation auc = 0.741784
Epoch 219
Loss = 2.2628e-01, PNorm = 43.7585, GNorm = 1.9857, lr_0 = 6.9548e-04
Validation auc = 0.753991
Epoch 220
Validation auc = 0.755399
Epoch 221
Validation auc = 0.755869
Epoch 222
Validation auc = 0.754930
Epoch 223
Validation auc = 0.750704
Epoch 224
Loss = 2.4273e-01, PNorm = 43.8471, GNorm = 1.8774, lr_0 = 6.8953e-04
Validation auc = 0.747887
Epoch 225
Validation auc = 0.753052
Epoch 226
Validation auc = 0.756338
Epoch 227
Validation auc = 0.753521
Epoch 228
Validation auc = 0.748357
Epoch 229
Loss = 2.7182e-01, PNorm = 43.9272, GNorm = 2.7130, lr_0 = 6.8364e-04
Validation auc = 0.742723
Epoch 230
Validation auc = 0.737559
Epoch 231
Validation auc = 0.736150
Epoch 232
Validation auc = 0.734742
Epoch 233
Validation auc = 0.732864
Epoch 234
Loss = 2.5148e-01, PNorm = 44.0142, GNorm = 1.9918, lr_0 = 6.7779e-04
Validation auc = 0.729577
Epoch 235
Validation auc = 0.730516
Epoch 236
Validation auc = 0.734742
Epoch 237
Validation auc = 0.736150
Epoch 238
Validation auc = 0.739437
Epoch 239
Loss = 2.0977e-01, PNorm = 44.1101, GNorm = 2.8347, lr_0 = 6.7200e-04
Validation auc = 0.733803
Epoch 240
Validation auc = 0.733333
Epoch 241
Validation auc = 0.732394
Epoch 242
Validation auc = 0.723474
Epoch 243
Validation auc = 0.730986
Epoch 244
Loss = 3.7964e-01, PNorm = 44.1979, GNorm = 2.6817, lr_0 = 6.6625e-04
Validation auc = 0.735211
Epoch 245
Validation auc = 0.732394
Epoch 246
Validation auc = 0.730986
Epoch 247
Validation auc = 0.734742
Epoch 248
Validation auc = 0.731925
Epoch 249
Loss = 3.3532e-01, PNorm = 44.2639, GNorm = 2.1740, lr_0 = 6.6056e-04
Validation auc = 0.736620
Epoch 250
Validation auc = 0.739437
Epoch 251
Validation auc = 0.738498
Epoch 252
Validation auc = 0.742254
Epoch 253
Validation auc = 0.746009
Epoch 254
Loss = 2.6617e-01, PNorm = 44.3378, GNorm = 2.7253, lr_0 = 6.5491e-04
Validation auc = 0.748357
Epoch 255
Validation auc = 0.750235
Epoch 256
Validation auc = 0.753052
Epoch 257
Validation auc = 0.754460
Epoch 258
Validation auc = 0.750704
Epoch 259
Loss = 4.4055e-01, PNorm = 44.4208, GNorm = 4.6089, lr_0 = 6.4931e-04
Validation auc = 0.747887
Epoch 260
Validation auc = 0.743192
Epoch 261
Validation auc = 0.737089
Epoch 262
Validation auc = 0.729108
Epoch 263
Validation auc = 0.723005
Epoch 264
Loss = 3.5164e-01, PNorm = 44.4848, GNorm = 2.3530, lr_0 = 6.4376e-04
Validation auc = 0.723474
Epoch 265
Validation auc = 0.723005
Epoch 266
Validation auc = 0.728169
Epoch 267
Validation auc = 0.731925
Epoch 268
Validation auc = 0.739906
Epoch 269
Loss = 2.2422e-01, PNorm = 44.5657, GNorm = 1.8320, lr_0 = 6.3826e-04
Validation auc = 0.746479
Epoch 270
Validation auc = 0.750704
Epoch 271
Validation auc = 0.747887
Epoch 272
Validation auc = 0.748826
Epoch 273
Validation auc = 0.750704
Epoch 274
Loss = 2.1286e-01, PNorm = 44.6566, GNorm = 1.4141, lr_0 = 6.3280e-04
Validation auc = 0.754460
Epoch 275
Validation auc = 0.753521
Epoch 276
Validation auc = 0.745540
Epoch 277
Validation auc = 0.740376
Epoch 278
Validation auc = 0.741784
Epoch 279
Loss = 2.7187e-01, PNorm = 44.7222, GNorm = 1.8185, lr_0 = 6.2739e-04
Validation auc = 0.747887
Epoch 280
Validation auc = 0.753052
Epoch 281
Validation auc = 0.752113
Epoch 282
Validation auc = 0.753991
Epoch 283
Validation auc = 0.754930
Epoch 284
Loss = 2.7825e-01, PNorm = 44.7941, GNorm = 4.0537, lr_0 = 6.2203e-04
Validation auc = 0.758216
Epoch 285
Validation auc = 0.757746
Epoch 286
Validation auc = 0.757277
Epoch 287
Validation auc = 0.746479
Epoch 288
Validation auc = 0.749765
Epoch 289
Loss = 2.1320e-01, PNorm = 44.8638, GNorm = 2.3243, lr_0 = 6.1671e-04
Validation auc = 0.747887
Epoch 290
Validation auc = 0.748826
Epoch 291
Validation auc = 0.754930
Epoch 292
Validation auc = 0.758685
Epoch 293
Validation auc = 0.750235
Epoch 294
Loss = 2.4331e-01, PNorm = 44.9449, GNorm = 2.9290, lr_0 = 6.1144e-04
Validation auc = 0.746009
Epoch 295
Validation auc = 0.744601
Epoch 296
Validation auc = 0.746479
Epoch 297
Validation auc = 0.742723
Epoch 298
Validation auc = 0.739437
Epoch 299
Loss = 2.3498e-01, PNorm = 45.0247, GNorm = 2.3303, lr_0 = 6.0621e-04
Validation auc = 0.735211
Model 0 best validation auc = 0.768545 on epoch 119
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.913305
Ensemble test auc = 0.913305
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.681221
Epoch 1
Validation auc = 0.677465
Epoch 2
Validation auc = 0.669953
Epoch 3
Validation auc = 0.662911
Epoch 4
Loss = 1.1727e+00, PNorm = 40.6676, GNorm = 14.6704, lr_0 = 6.5000e-04
Validation auc = 0.638028
Epoch 5
Validation auc = 0.654930
Epoch 6
Validation auc = 0.653991
Epoch 7
Validation auc = 0.648357
Epoch 8
Validation auc = 0.649296
Epoch 9
Loss = 8.6141e-01, PNorm = 40.7109, GNorm = 9.1591, lr_0 = 9.9743e-04
Validation auc = 0.627230
Epoch 10
Validation auc = 0.619718
Epoch 11
Validation auc = 0.644601
Epoch 12
Validation auc = 0.622535
Epoch 13
Validation auc = 0.618310
Epoch 14
Loss = 7.2814e-01, PNorm = 40.7650, GNorm = 6.6048, lr_0 = 9.8890e-04
Validation auc = 0.599061
Epoch 15
Validation auc = 0.637089
Epoch 16
Validation auc = 0.651174
Epoch 17
Validation auc = 0.652582
Epoch 18
Validation auc = 0.724413
Epoch 19
Loss = 6.7291e-01, PNorm = 40.8132, GNorm = 6.3667, lr_0 = 9.8045e-04
Validation auc = 0.657746
Epoch 20
Validation auc = 0.662441
Epoch 21
Validation auc = 0.662911
Epoch 22
Validation auc = 0.681690
Epoch 23
Validation auc = 0.687324
Epoch 24
Loss = 7.9424e-01, PNorm = 40.8495, GNorm = 7.9910, lr_0 = 9.7207e-04
Validation auc = 0.672300
Epoch 25
Validation auc = 0.674178
Epoch 26
Validation auc = 0.658685
Epoch 27
Validation auc = 0.655399
Epoch 28
Validation auc = 0.656338
Epoch 29
Loss = 5.4907e-01, PNorm = 40.8834, GNorm = 4.3092, lr_0 = 9.6376e-04
Validation auc = 0.656808
Epoch 30
Validation auc = 0.641784
Epoch 31
Validation auc = 0.609859
Epoch 32
Validation auc = 0.600469
Epoch 33
Validation auc = 0.607981
Epoch 34
Loss = 7.4850e-01, PNorm = 40.9016, GNorm = 6.1751, lr_0 = 9.5552e-04
Validation auc = 0.623474
Epoch 35
Validation auc = 0.663850
Epoch 36
Validation auc = 0.698592
Epoch 37
Validation auc = 0.701878
Epoch 38
Validation auc = 0.703756
Epoch 39
Loss = 6.8735e-01, PNorm = 40.9312, GNorm = 3.6573, lr_0 = 9.4735e-04
Validation auc = 0.728638
Epoch 40
Validation auc = 0.763850
Epoch 41
Validation auc = 0.732394
Epoch 42
Validation auc = 0.700469
Epoch 43
Validation auc = 0.695775
Epoch 44
Loss = 6.6924e-01, PNorm = 40.9551, GNorm = 5.3660, lr_0 = 9.3925e-04
Validation auc = 0.681221
Epoch 45
Validation auc = 0.681690
Epoch 46
Validation auc = 0.678404
Epoch 47
Validation auc = 0.654930
Epoch 48
Validation auc = 0.659624
Epoch 49
Loss = 6.4416e-01, PNorm = 40.9989, GNorm = 4.5010, lr_0 = 9.3122e-04
Validation auc = 0.665258
Epoch 50
Validation auc = 0.669484
Epoch 51
Validation auc = 0.659155
Epoch 52
Validation auc = 0.645070
Epoch 53
Validation auc = 0.623944
Epoch 54
Loss = 5.9989e-01, PNorm = 41.0301, GNorm = 5.4409, lr_0 = 9.2326e-04
Validation auc = 0.617371
Epoch 55
Validation auc = 0.619718
Epoch 56
Validation auc = 0.618310
Epoch 57
Validation auc = 0.618310
Epoch 58
Validation auc = 0.631455
Epoch 59
Loss = 5.9493e-01, PNorm = 41.0679, GNorm = 4.3275, lr_0 = 9.1537e-04
Validation auc = 0.637089
Epoch 60
Validation auc = 0.631925
Epoch 61
Validation auc = 0.631455
Epoch 62
Validation auc = 0.641315
Epoch 63
Validation auc = 0.656338
Epoch 64
Loss = 6.0904e-01, PNorm = 41.1015, GNorm = 3.8266, lr_0 = 9.0754e-04
Validation auc = 0.650235
Epoch 65
Validation auc = 0.646009
Epoch 66
Validation auc = 0.636620
Epoch 67
Validation auc = 0.631925
Epoch 68
Validation auc = 0.638028
Epoch 69
Loss = 4.7100e-01, PNorm = 41.1589, GNorm = 3.7963, lr_0 = 8.9978e-04
Validation auc = 0.646479
Epoch 70
Validation auc = 0.649765
Epoch 71
Validation auc = 0.637559
Epoch 72
Validation auc = 0.633333
Epoch 73
Validation auc = 0.631455
Epoch 74
Loss = 5.6367e-01, PNorm = 41.2120, GNorm = 4.3209, lr_0 = 8.9209e-04
Validation auc = 0.630516
Epoch 75
Validation auc = 0.634742
Epoch 76
Validation auc = 0.641315
Epoch 77
Validation auc = 0.637089
Epoch 78
Validation auc = 0.634742
Epoch 79
Loss = 5.2709e-01, PNorm = 41.2384, GNorm = 3.4510, lr_0 = 8.8447e-04
Validation auc = 0.626761
Epoch 80
Validation auc = 0.624883
Epoch 81
Validation auc = 0.625822
Epoch 82
Validation auc = 0.630516
Epoch 83
Validation auc = 0.636620
Epoch 84
Loss = 4.4689e-01, PNorm = 41.3026, GNorm = 2.6538, lr_0 = 8.7691e-04
Validation auc = 0.635681
Epoch 85
Validation auc = 0.646479
Epoch 86
Validation auc = 0.648826
Epoch 87
Validation auc = 0.647418
Epoch 88
Validation auc = 0.653052
Epoch 89
Loss = 5.0990e-01, PNorm = 41.3538, GNorm = 3.5241, lr_0 = 8.6941e-04
Validation auc = 0.654930
Epoch 90
Validation auc = 0.658685
Epoch 91
Validation auc = 0.666197
Epoch 92
Validation auc = 0.665258
Epoch 93
Validation auc = 0.669953
Epoch 94
Loss = 4.5925e-01, PNorm = 41.4165, GNorm = 2.3186, lr_0 = 8.6198e-04
Validation auc = 0.666667
Epoch 95
Validation auc = 0.661033
Epoch 96
Validation auc = 0.647887
Epoch 97
Validation auc = 0.637559
Epoch 98
Validation auc = 0.630047
Epoch 99
Loss = 5.3186e-01, PNorm = 41.4778, GNorm = 3.0300, lr_0 = 8.5461e-04
Validation auc = 0.626761
Epoch 100
Validation auc = 0.625822
Epoch 101
Validation auc = 0.619718
Epoch 102
Validation auc = 0.628169
Epoch 103
Validation auc = 0.625822
Epoch 104
Loss = 5.0644e-01, PNorm = 41.5647, GNorm = 2.6269, lr_0 = 8.4730e-04
Validation auc = 0.626291
Epoch 105
Validation auc = 0.631925
Epoch 106
Validation auc = 0.637559
Epoch 107
Validation auc = 0.650235
Epoch 108
Validation auc = 0.660094
Epoch 109
Loss = 4.2352e-01, PNorm = 41.6538, GNorm = 2.4346, lr_0 = 8.4006e-04
Validation auc = 0.651643
Epoch 110
Validation auc = 0.643662
Epoch 111
Validation auc = 0.650235
Epoch 112
Validation auc = 0.652113
Epoch 113
Validation auc = 0.646009
Epoch 114
Loss = 5.1174e-01, PNorm = 41.7315, GNorm = 2.9920, lr_0 = 8.3288e-04
Validation auc = 0.652113
Epoch 115
Validation auc = 0.652113
Epoch 116
Validation auc = 0.646479
Epoch 117
Validation auc = 0.649765
Epoch 118
Validation auc = 0.650704
Epoch 119
Loss = 4.2323e-01, PNorm = 41.8124, GNorm = 2.3519, lr_0 = 8.2576e-04
Validation auc = 0.649296
Epoch 120
Validation auc = 0.647418
Epoch 121
Validation auc = 0.648826
Epoch 122
Validation auc = 0.643662
Epoch 123
Validation auc = 0.640376
Epoch 124
Loss = 4.2543e-01, PNorm = 41.8969, GNorm = 2.5333, lr_0 = 8.1870e-04
Validation auc = 0.644131
Epoch 125
Validation auc = 0.660094
Epoch 126
Validation auc = 0.658216
Epoch 127
Validation auc = 0.659624
Epoch 128
Validation auc = 0.659624
Epoch 129
Loss = 4.1787e-01, PNorm = 41.9872, GNorm = 1.8893, lr_0 = 8.1170e-04
Validation auc = 0.668075
Epoch 130
Validation auc = 0.680282
Epoch 131
Validation auc = 0.689671
Epoch 132
Validation auc = 0.694836
Epoch 133
Validation auc = 0.691549
Epoch 134
Loss = 5.1003e-01, PNorm = 42.0893, GNorm = 4.1963, lr_0 = 8.0476e-04
Validation auc = 0.676056
Epoch 135
Validation auc = 0.681221
Epoch 136
Validation auc = 0.684507
Epoch 137
Validation auc = 0.676526
Epoch 138
Validation auc = 0.679812
Epoch 139
Loss = 4.4795e-01, PNorm = 42.1844, GNorm = 2.8275, lr_0 = 7.9788e-04
Validation auc = 0.677465
Epoch 140
Validation auc = 0.690610
Epoch 141
Validation auc = 0.702347
Epoch 142
Validation auc = 0.684977
Epoch 143
Validation auc = 0.684507
Epoch 144
Loss = 4.3959e-01, PNorm = 42.2816, GNorm = 2.5273, lr_0 = 7.9106e-04
Validation auc = 0.687324
Epoch 145
Validation auc = 0.692019
Epoch 146
Validation auc = 0.710798
Epoch 147
Validation auc = 0.718779
Epoch 148
Validation auc = 0.721596
Epoch 149
Loss = 3.9867e-01, PNorm = 42.3744, GNorm = 2.1225, lr_0 = 7.8430e-04
Validation auc = 0.719718
Epoch 150
Validation auc = 0.723944
Epoch 151
Validation auc = 0.734272
Epoch 152
Validation auc = 0.740376
Epoch 153
Validation auc = 0.748826
Epoch 154
Loss = 5.2541e-01, PNorm = 42.4580, GNorm = 3.9638, lr_0 = 7.7759e-04
Validation auc = 0.757277
Epoch 155
Validation auc = 0.744601
Epoch 156
Validation auc = 0.729577
Epoch 157
Validation auc = 0.718779
Epoch 158
Validation auc = 0.713146
Epoch 159
Loss = 4.0985e-01, PNorm = 42.5545, GNorm = 2.7254, lr_0 = 7.7095e-04
Validation auc = 0.711737
Epoch 160
Validation auc = 0.718310
Epoch 161
Validation auc = 0.725822
Epoch 162
Validation auc = 0.731925
Epoch 163
Validation auc = 0.733333
Epoch 164
Loss = 4.2791e-01, PNorm = 42.6407, GNorm = 2.3567, lr_0 = 7.6436e-04
Validation auc = 0.730047
Epoch 165
Validation auc = 0.719718
Epoch 166
Validation auc = 0.715962
Epoch 167
Validation auc = 0.708920
Epoch 168
Validation auc = 0.703756
Epoch 169
Loss = 4.0718e-01, PNorm = 42.7321, GNorm = 2.4523, lr_0 = 7.5782e-04
Validation auc = 0.705164
Epoch 170
Validation auc = 0.700939
Epoch 171
Validation auc = 0.708920
Epoch 172
Validation auc = 0.715493
Epoch 173
Validation auc = 0.732394
Epoch 174
Loss = 4.5649e-01, PNorm = 42.8361, GNorm = 3.3361, lr_0 = 7.5134e-04
Validation auc = 0.737089
Epoch 175
Validation auc = 0.735211
Epoch 176
Validation auc = 0.730047
Epoch 177
Validation auc = 0.723944
Epoch 178
Validation auc = 0.707512
Epoch 179
Loss = 4.6537e-01, PNorm = 42.9396, GNorm = 3.4402, lr_0 = 7.4492e-04
Validation auc = 0.700469
Epoch 180
Validation auc = 0.704695
Epoch 181
Validation auc = 0.706573
Epoch 182
Validation auc = 0.722535
Epoch 183
Validation auc = 0.742723
Epoch 184
Loss = 4.4325e-01, PNorm = 43.0437, GNorm = 3.9401, lr_0 = 7.3855e-04
Validation auc = 0.746948
Epoch 185
Validation auc = 0.736620
Epoch 186
Validation auc = 0.716901
Epoch 187
Validation auc = 0.703286
Epoch 188
Validation auc = 0.693897
Epoch 189
Loss = 3.7391e-01, PNorm = 43.1419, GNorm = 2.0179, lr_0 = 7.3224e-04
Validation auc = 0.689671
Epoch 190
Validation auc = 0.697653
Epoch 191
Validation auc = 0.697653
Epoch 192
Validation auc = 0.702817
Epoch 193
Validation auc = 0.711268
Epoch 194
Loss = 3.6545e-01, PNorm = 43.2292, GNorm = 3.6288, lr_0 = 7.2598e-04
Validation auc = 0.723944
Epoch 195
Validation auc = 0.732864
Epoch 196
Validation auc = 0.732864
Epoch 197
Validation auc = 0.725822
Epoch 198
Validation auc = 0.725822
Epoch 199
Loss = 3.3024e-01, PNorm = 43.3228, GNorm = 2.1905, lr_0 = 7.1977e-04
Validation auc = 0.723005
Epoch 200
Validation auc = 0.724883
Epoch 201
Validation auc = 0.726291
Epoch 202
Validation auc = 0.727700
Epoch 203
Validation auc = 0.730047
Epoch 204
Loss = 4.2872e-01, PNorm = 43.4180, GNorm = 1.7103, lr_0 = 7.1362e-04
Validation auc = 0.731925
Epoch 205
Validation auc = 0.738028
Epoch 206
Validation auc = 0.738028
Epoch 207
Validation auc = 0.731925
Epoch 208
Validation auc = 0.725352
Epoch 209
Loss = 3.1306e-01, PNorm = 43.5044, GNorm = 2.8695, lr_0 = 7.0752e-04
Validation auc = 0.717840
Epoch 210
Validation auc = 0.711737
Epoch 211
Validation auc = 0.708920
Epoch 212
Validation auc = 0.713615
Epoch 213
Validation auc = 0.710329
Epoch 214
Loss = 2.8018e-01, PNorm = 43.5930, GNorm = 2.2115, lr_0 = 7.0147e-04
Validation auc = 0.704695
Epoch 215
Validation auc = 0.707512
Epoch 216
Validation auc = 0.710798
Epoch 217
Validation auc = 0.714554
Epoch 218
Validation auc = 0.715962
Epoch 219
Loss = 2.9224e-01, PNorm = 43.6914, GNorm = 2.0517, lr_0 = 6.9548e-04
Validation auc = 0.723474
Epoch 220
Validation auc = 0.723005
Epoch 221
Validation auc = 0.723005
Epoch 222
Validation auc = 0.719249
Epoch 223
Validation auc = 0.720188
Epoch 224
Loss = 3.1945e-01, PNorm = 43.7872, GNorm = 2.2772, lr_0 = 6.8953e-04
Validation auc = 0.723944
Epoch 225
Validation auc = 0.719249
Epoch 226
Validation auc = 0.716432
Epoch 227
Validation auc = 0.713146
Epoch 228
Validation auc = 0.708451
Epoch 229
Loss = 3.7921e-01, PNorm = 43.8907, GNorm = 3.8619, lr_0 = 6.8364e-04
Validation auc = 0.713146
Epoch 230
Validation auc = 0.720657
Epoch 231
Validation auc = 0.733333
Epoch 232
Validation auc = 0.748357
Epoch 233
Validation auc = 0.745070
Epoch 234
Loss = 2.9535e-01, PNorm = 43.9904, GNorm = 2.2146, lr_0 = 6.7779e-04
Validation auc = 0.729577
Epoch 235
Validation auc = 0.710798
Epoch 236
Validation auc = 0.710329
Epoch 237
Validation auc = 0.714085
Epoch 238
Validation auc = 0.710329
Epoch 239
Loss = 3.3344e-01, PNorm = 44.0809, GNorm = 3.4018, lr_0 = 6.7200e-04
Validation auc = 0.714554
Epoch 240
Validation auc = 0.722535
Epoch 241
Validation auc = 0.728169
Epoch 242
Validation auc = 0.738028
Epoch 243
Validation auc = 0.733333
Epoch 244
Loss = 2.3145e-01, PNorm = 44.1781, GNorm = 2.0004, lr_0 = 6.6625e-04
Validation auc = 0.724883
Epoch 245
Validation auc = 0.713615
Epoch 246
Validation auc = 0.713615
Epoch 247
Validation auc = 0.711737
Epoch 248
Validation auc = 0.712676
Epoch 249
Loss = 2.8079e-01, PNorm = 44.2733, GNorm = 2.0464, lr_0 = 6.6056e-04
Validation auc = 0.710798
Epoch 250
Validation auc = 0.715962
Epoch 251
Validation auc = 0.723474
Epoch 252
Validation auc = 0.732394
Epoch 253
Validation auc = 0.731925
Epoch 254
Loss = 2.6655e-01, PNorm = 44.3600, GNorm = 3.6381, lr_0 = 6.5491e-04
Validation auc = 0.727230
Epoch 255
Validation auc = 0.723474
Epoch 256
Validation auc = 0.722066
Epoch 257
Validation auc = 0.723005
Epoch 258
Validation auc = 0.728169
Epoch 259
Loss = 2.9938e-01, PNorm = 44.4516, GNorm = 3.5554, lr_0 = 6.4931e-04
Validation auc = 0.730047
Epoch 260
Validation auc = 0.732394
Epoch 261
Validation auc = 0.735211
Epoch 262
Validation auc = 0.741784
Epoch 263
Validation auc = 0.739906
Epoch 264
Loss = 3.0951e-01, PNorm = 44.5336, GNorm = 4.1220, lr_0 = 6.4376e-04
Validation auc = 0.734742
Epoch 265
Validation auc = 0.727700
Epoch 266
Validation auc = 0.721127
Epoch 267
Validation auc = 0.720188
Epoch 268
Validation auc = 0.721127
Epoch 269
Loss = 3.5341e-01, PNorm = 44.6227, GNorm = 3.4183, lr_0 = 6.3826e-04
Validation auc = 0.720657
Epoch 270
Validation auc = 0.720188
Epoch 271
Validation auc = 0.719718
Epoch 272
Validation auc = 0.726291
Epoch 273
Validation auc = 0.724413
Epoch 274
Loss = 3.0888e-01, PNorm = 44.7005, GNorm = 2.2725, lr_0 = 6.3280e-04
Validation auc = 0.727230
Epoch 275
Validation auc = 0.729577
Epoch 276
Validation auc = 0.731455
Epoch 277
Validation auc = 0.720657
Epoch 278
Validation auc = 0.723474
Epoch 279
Loss = 2.8115e-01, PNorm = 44.7759, GNorm = 2.6436, lr_0 = 6.2739e-04
Validation auc = 0.727700
Epoch 280
Validation auc = 0.725352
Epoch 281
Validation auc = 0.723005
Epoch 282
Validation auc = 0.722535
Epoch 283
Validation auc = 0.729577
Epoch 284
Loss = 2.7123e-01, PNorm = 44.8542, GNorm = 2.2509, lr_0 = 6.2203e-04
Validation auc = 0.730047
Epoch 285
Validation auc = 0.736620
Epoch 286
Validation auc = 0.742254
Epoch 287
Validation auc = 0.733803
Epoch 288
Validation auc = 0.727700
Epoch 289
Loss = 2.2580e-01, PNorm = 44.9367, GNorm = 1.7475, lr_0 = 6.1671e-04
Validation auc = 0.723474
Epoch 290
Validation auc = 0.717840
Epoch 291
Validation auc = 0.715493
Epoch 292
Validation auc = 0.711737
Epoch 293
Validation auc = 0.708920
Epoch 294
Loss = 2.3464e-01, PNorm = 45.0167, GNorm = 2.4002, lr_0 = 6.1144e-04
Validation auc = 0.716432
Epoch 295
Validation auc = 0.723944
Epoch 296
Validation auc = 0.730047
Epoch 297
Validation auc = 0.738028
Epoch 298
Validation auc = 0.737559
Epoch 299
Loss = 2.5384e-01, PNorm = 45.1094, GNorm = 1.6387, lr_0 = 6.0621e-04
Validation auc = 0.731455
Model 0 best validation auc = 0.763850 on epoch 40
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.852498
Ensemble test auc = 0.852498
10-fold cross validation
	Seed 0 ==> test auc = 0.856111
	Seed 1 ==> test auc = 0.779049
	Seed 2 ==> test auc = 0.921734
	Seed 3 ==> test auc = 0.797110
	Seed 4 ==> test auc = 0.918724
	Seed 5 ==> test auc = 0.919326
	Seed 6 ==> test auc = 0.913305
	Seed 7 ==> test auc = 0.927152
	Seed 8 ==> test auc = 0.913305
	Seed 9 ==> test auc = 0.852498
Overall test auc = 0.879831 +/- 0.052553
Elapsed time = 0:04:55
Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=1),
 'dropout': 0.30000000000000004,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 300,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 300,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.628169
Epoch 1
Validation auc = 0.695305
Epoch 2
Validation auc = 0.695775
Epoch 3
Validation auc = 0.698592
Epoch 4
Loss = 1.0586e+00, PNorm = 40.6669, GNorm = 12.6522, lr_0 = 6.5000e-04
Validation auc = 0.730986
Epoch 5
Validation auc = 0.719718
Epoch 6
Validation auc = 0.658685
Epoch 7
Validation auc = 0.630986
Epoch 8
Validation auc = 0.614554
Epoch 9
Loss = 9.4697e-01, PNorm = 40.7117, GNorm = 11.4273, lr_0 = 9.9743e-04
Validation auc = 0.592488
Epoch 10
Validation auc = 0.669484
Epoch 11
Validation auc = 0.636150
Epoch 12
Validation auc = 0.654460
Epoch 13
Validation auc = 0.662441
Epoch 14
Loss = 8.9000e-01, PNorm = 40.7700, GNorm = 8.3154, lr_0 = 9.8890e-04
Validation auc = 0.714085
Epoch 15
Validation auc = 0.723474
Epoch 16
Validation auc = 0.698122
Epoch 17
Validation auc = 0.689671
Epoch 18
Validation auc = 0.692488
Epoch 19
Loss = 6.4386e-01, PNorm = 40.8093, GNorm = 5.7968, lr_0 = 9.8045e-04
Validation auc = 0.713146
Epoch 20
Validation auc = 0.737089
Epoch 21
Validation auc = 0.731925
Epoch 22
Validation auc = 0.715493
Epoch 23
Validation auc = 0.715962
Epoch 24
Loss = 7.5890e-01, PNorm = 40.8456, GNorm = 4.7717, lr_0 = 9.7207e-04
Validation auc = 0.718310
Epoch 25
Validation auc = 0.731925
Epoch 26
Validation auc = 0.747887
Epoch 27
Validation auc = 0.760094
Epoch 28
Validation auc = 0.762911
Epoch 29
Loss = 6.4495e-01, PNorm = 40.8848, GNorm = 5.2614, lr_0 = 9.6376e-04
Validation auc = 0.753052
Epoch 30
Validation auc = 0.729577
Epoch 31
Validation auc = 0.701408
Epoch 32
Validation auc = 0.689671
Epoch 33
Validation auc = 0.690610
Epoch 34
Loss = 6.2325e-01, PNorm = 40.9212, GNorm = 6.1661, lr_0 = 9.5552e-04
Validation auc = 0.687793
Epoch 35
Validation auc = 0.690141
Epoch 36
Validation auc = 0.678404
Epoch 37
Validation auc = 0.668545
Epoch 38
Validation auc = 0.644601
Epoch 39
Loss = 5.5793e-01, PNorm = 40.9536, GNorm = 3.7693, lr_0 = 9.4735e-04
Validation auc = 0.650235
Epoch 40
Validation auc = 0.648357
Epoch 41
Validation auc = 0.670423
Epoch 42
Validation auc = 0.678873
Epoch 43
Validation auc = 0.690141
Epoch 44
Loss = 5.1738e-01, PNorm = 40.9941, GNorm = 4.0626, lr_0 = 9.3925e-04
Validation auc = 0.697653
Epoch 45
Validation auc = 0.684038
Epoch 46
Validation auc = 0.674648
Epoch 47
Validation auc = 0.668545
Epoch 48
Validation auc = 0.652113
Epoch 49
Loss = 6.5326e-01, PNorm = 41.0340, GNorm = 5.2204, lr_0 = 9.3122e-04
Validation auc = 0.645540
Epoch 50
Validation auc = 0.654930
Epoch 51
Validation auc = 0.657277
Epoch 52
Validation auc = 0.657277
Epoch 53
Validation auc = 0.651643
Epoch 54
Loss = 5.4901e-01, PNorm = 41.0795, GNorm = 5.6308, lr_0 = 9.2326e-04
Validation auc = 0.646948
Epoch 55
Validation auc = 0.646948
Epoch 56
Validation auc = 0.648826
Epoch 57
Validation auc = 0.649296
Epoch 58
Validation auc = 0.641784
Epoch 59
Loss = 6.5911e-01, PNorm = 41.1254, GNorm = 5.2338, lr_0 = 9.1537e-04
Validation auc = 0.638967
Epoch 60
Validation auc = 0.642254
Epoch 61
Validation auc = 0.638028
Epoch 62
Validation auc = 0.641784
Epoch 63
Validation auc = 0.648357
Epoch 64
Loss = 5.2210e-01, PNorm = 41.1934, GNorm = 4.7278, lr_0 = 9.0754e-04
Validation auc = 0.646479
Epoch 65
Validation auc = 0.641784
Epoch 66
Validation auc = 0.622535
Epoch 67
Validation auc = 0.616901
Epoch 68
Validation auc = 0.618779
Epoch 69
Loss = 4.8703e-01, PNorm = 41.2472, GNorm = 3.2865, lr_0 = 8.9978e-04
Validation auc = 0.628638
Epoch 70
Validation auc = 0.652582
Epoch 71
Validation auc = 0.642723
Epoch 72
Validation auc = 0.647887
Epoch 73
Validation auc = 0.642723
Epoch 74
Loss = 6.0897e-01, PNorm = 41.3447, GNorm = 4.1058, lr_0 = 8.9209e-04
Validation auc = 0.634272
Epoch 75
Validation auc = 0.629108
Epoch 76
Validation auc = 0.628169
Epoch 77
Validation auc = 0.620188
Epoch 78
Validation auc = 0.615962
Epoch 79
Loss = 6.5077e-01, PNorm = 41.4027, GNorm = 4.8542, lr_0 = 8.8447e-04
Validation auc = 0.621596
Epoch 80
Validation auc = 0.618779
Epoch 81
Validation auc = 0.621596
Epoch 82
Validation auc = 0.629108
Epoch 83
Validation auc = 0.631925
Epoch 84
Loss = 4.4870e-01, PNorm = 41.4659, GNorm = 2.9592, lr_0 = 8.7691e-04
Validation auc = 0.630516
Epoch 85
Validation auc = 0.633803
Epoch 86
Validation auc = 0.636150
Epoch 87
Validation auc = 0.635681
Epoch 88
Validation auc = 0.646479
Epoch 89
Loss = 5.3074e-01, PNorm = 41.5356, GNorm = 4.7615, lr_0 = 8.6941e-04
Validation auc = 0.656338
Epoch 90
Validation auc = 0.653052
Epoch 91
Validation auc = 0.650704
Epoch 92
Validation auc = 0.653521
Epoch 93
Validation auc = 0.657277
Epoch 94
Loss = 5.0271e-01, PNorm = 41.5936, GNorm = 2.9955, lr_0 = 8.6198e-04
Validation auc = 0.681221
Epoch 95
Validation auc = 0.676526
Epoch 96
Validation auc = 0.652582
Epoch 97
Validation auc = 0.641315
Epoch 98
Validation auc = 0.642723
Epoch 99
Loss = 4.0967e-01, PNorm = 41.6846, GNorm = 2.3018, lr_0 = 8.5461e-04
Validation auc = 0.651174
Epoch 100
Validation auc = 0.656808
Epoch 101
Validation auc = 0.656338
Epoch 102
Validation auc = 0.665258
Epoch 103
Validation auc = 0.676526
Epoch 104
Loss = 5.1816e-01, PNorm = 41.7601, GNorm = 4.4473, lr_0 = 8.4730e-04
Validation auc = 0.679812
Epoch 105
Validation auc = 0.683099
Epoch 106
Validation auc = 0.672300
Epoch 107
Validation auc = 0.668545
Epoch 108
Validation auc = 0.668075
Epoch 109
Loss = 5.3604e-01, PNorm = 41.8502, GNorm = 2.7722, lr_0 = 8.4006e-04
Validation auc = 0.669484
Epoch 110
Validation auc = 0.665258
Epoch 111
Validation auc = 0.668075
Epoch 112
Validation auc = 0.667606
Epoch 113
Validation auc = 0.666197
Epoch 114
Loss = 4.8188e-01, PNorm = 41.9245, GNorm = 2.7175, lr_0 = 8.3288e-04
Validation auc = 0.670892
Epoch 115
Validation auc = 0.668545
Epoch 116
Validation auc = 0.669484
Epoch 117
Validation auc = 0.670892
Epoch 118
Validation auc = 0.675117
Epoch 119
Loss = 4.4285e-01, PNorm = 42.0460, GNorm = 4.2058, lr_0 = 8.2576e-04
Validation auc = 0.684977
Epoch 120
Validation auc = 0.700000
Epoch 121
Validation auc = 0.707042
Epoch 122
Validation auc = 0.706573
Epoch 123
Validation auc = 0.700939
Epoch 124
Loss = 4.2226e-01, PNorm = 42.1521, GNorm = 3.8331, lr_0 = 8.1870e-04
Validation auc = 0.684977
Epoch 125
Validation auc = 0.676056
Epoch 126
Validation auc = 0.674648
Epoch 127
Validation auc = 0.678873
Epoch 128
Validation auc = 0.672770
Epoch 129
Loss = 3.5606e-01, PNorm = 42.2504, GNorm = 1.9712, lr_0 = 8.1170e-04
Validation auc = 0.659155
Epoch 130
Validation auc = 0.666197
Epoch 131
Validation auc = 0.673709
Epoch 132
Validation auc = 0.673709
Epoch 133
Validation auc = 0.666197
Epoch 134
Loss = 4.2385e-01, PNorm = 42.3388, GNorm = 3.3083, lr_0 = 8.0476e-04
Validation auc = 0.653521
Epoch 135
Validation auc = 0.656338
Epoch 136
Validation auc = 0.672770
Epoch 137
Validation auc = 0.689202
Epoch 138
Validation auc = 0.688263
Epoch 139
Loss = 3.8030e-01, PNorm = 42.4286, GNorm = 2.9730, lr_0 = 7.9788e-04
Validation auc = 0.687324
Epoch 140
Validation auc = 0.697653
Epoch 141
Validation auc = 0.707512
Epoch 142
Validation auc = 0.718310
Epoch 143
Validation auc = 0.723944
Epoch 144
Loss = 4.4222e-01, PNorm = 42.5211, GNorm = 2.4797, lr_0 = 7.9106e-04
Validation auc = 0.722535
Epoch 145
Validation auc = 0.716901
Epoch 146
Validation auc = 0.715023
Epoch 147
Validation auc = 0.717371
Epoch 148
Validation auc = 0.716901
Epoch 149
Loss = 3.7644e-01, PNorm = 42.6240, GNorm = 3.6333, lr_0 = 7.8430e-04
Validation auc = 0.711268
Epoch 150
Validation auc = 0.717371
Epoch 151
Validation auc = 0.723474
Epoch 152
Validation auc = 0.728169
Epoch 153
Validation auc = 0.736150
Epoch 154
Loss = 4.3723e-01, PNorm = 42.7339, GNorm = 2.5887, lr_0 = 7.7759e-04
Validation auc = 0.749765
Epoch 155
Validation auc = 0.746948
Epoch 156
Validation auc = 0.740376
Epoch 157
Validation auc = 0.736150
Epoch 158
Validation auc = 0.726761
Epoch 159
Loss = 3.6100e-01, PNorm = 42.8272, GNorm = 3.5246, lr_0 = 7.7095e-04
Validation auc = 0.722066
Epoch 160
Validation auc = 0.715023
Epoch 161
Validation auc = 0.711737
Epoch 162
Validation auc = 0.722535
Epoch 163
Validation auc = 0.727230
Epoch 164
Loss = 4.1562e-01, PNorm = 42.9251, GNorm = 2.8812, lr_0 = 7.6436e-04
Validation auc = 0.731455
Epoch 165
Validation auc = 0.735681
Epoch 166
Validation auc = 0.729577
Epoch 167
Validation auc = 0.715493
Epoch 168
Validation auc = 0.701878
Epoch 169
Loss = 3.3769e-01, PNorm = 43.0134, GNorm = 3.8247, lr_0 = 7.5782e-04
Validation auc = 0.699531
Epoch 170
Validation auc = 0.697653
Epoch 171
Validation auc = 0.703286
Epoch 172
Validation auc = 0.700939
Epoch 173
Validation auc = 0.704225
Epoch 174
Loss = 3.7853e-01, PNorm = 43.1116, GNorm = 2.9030, lr_0 = 7.5134e-04
Validation auc = 0.715023
Epoch 175
Validation auc = 0.716901
Epoch 176
Validation auc = 0.728638
Epoch 177
Validation auc = 0.728638
Epoch 178
Validation auc = 0.727700
Epoch 179
Loss = 3.6827e-01, PNorm = 43.2310, GNorm = 2.7030, lr_0 = 7.4492e-04
Validation auc = 0.729577
Epoch 180
Validation auc = 0.729577
Epoch 181
Validation auc = 0.710798
Epoch 182
Validation auc = 0.704695
Epoch 183
Validation auc = 0.704225
Epoch 184
Loss = 3.2601e-01, PNorm = 43.3343, GNorm = 2.4061, lr_0 = 7.3855e-04
Validation auc = 0.709859
Epoch 185
Validation auc = 0.715023
Epoch 186
Validation auc = 0.715493
Epoch 187
Validation auc = 0.711268
Epoch 188
Validation auc = 0.712207
Epoch 189
Loss = 4.9626e-01, PNorm = 43.4414, GNorm = 4.7753, lr_0 = 7.3224e-04
Validation auc = 0.719249
Epoch 190
Validation auc = 0.730516
Epoch 191
Validation auc = 0.730986
Epoch 192
Validation auc = 0.726761
Epoch 193
Validation auc = 0.723944
Epoch 194
Loss = 3.4785e-01, PNorm = 43.5355, GNorm = 2.0181, lr_0 = 7.2598e-04
Validation auc = 0.723005
Epoch 195
Validation auc = 0.734272
Epoch 196
Validation auc = 0.734272
Epoch 197
Validation auc = 0.731455
Epoch 198
Validation auc = 0.733333
Epoch 199
Loss = 2.9465e-01, PNorm = 43.6278, GNorm = 2.5281, lr_0 = 7.1977e-04
Validation auc = 0.729577
Epoch 200
Validation auc = 0.723474
Epoch 201
Validation auc = 0.728638
Epoch 202
Validation auc = 0.730516
Epoch 203
Validation auc = 0.730047
Epoch 204
Loss = 3.0448e-01, PNorm = 43.7115, GNorm = 2.9157, lr_0 = 7.1362e-04
Validation auc = 0.728169
Epoch 205
Validation auc = 0.727700
Epoch 206
Validation auc = 0.729108
Epoch 207
Validation auc = 0.730047
Epoch 208
Validation auc = 0.735681
Epoch 209
Loss = 2.1663e-01, PNorm = 43.8098, GNorm = 2.9022, lr_0 = 7.0752e-04
Validation auc = 0.734742
Epoch 210
Validation auc = 0.734272
Epoch 211
Validation auc = 0.730516
Epoch 212
Validation auc = 0.730516
Epoch 213
Validation auc = 0.730047
Epoch 214
Loss = 3.1711e-01, PNorm = 43.9030, GNorm = 3.3697, lr_0 = 7.0147e-04
Validation auc = 0.729108
Epoch 215
Validation auc = 0.727230
Epoch 216
Validation auc = 0.720188
Epoch 217
Validation auc = 0.715493
Epoch 218
Validation auc = 0.715962
Epoch 219
Loss = 2.7173e-01, PNorm = 43.9853, GNorm = 2.8156, lr_0 = 6.9548e-04
Validation auc = 0.718310
Epoch 220
Validation auc = 0.717840
Epoch 221
Validation auc = 0.720657
Epoch 222
Validation auc = 0.721596
Epoch 223
Validation auc = 0.727230
Epoch 224
Loss = 2.6804e-01, PNorm = 44.0681, GNorm = 3.0562, lr_0 = 6.8953e-04
Validation auc = 0.730047
Epoch 225
Validation auc = 0.736150
Epoch 226
Validation auc = 0.735211
Epoch 227
Validation auc = 0.731455
Epoch 228
Validation auc = 0.723474
Epoch 229
Loss = 3.6117e-01, PNorm = 44.1421, GNorm = 2.2142, lr_0 = 6.8364e-04
Validation auc = 0.730986
Epoch 230
Validation auc = 0.729108
Epoch 231
Validation auc = 0.727230
Epoch 232
Validation auc = 0.725352
Epoch 233
Validation auc = 0.724883
Epoch 234
Loss = 2.2079e-01, PNorm = 44.2325, GNorm = 1.7093, lr_0 = 6.7779e-04
Validation auc = 0.723944
Epoch 235
Validation auc = 0.725822
Epoch 236
Validation auc = 0.723944
Epoch 237
Validation auc = 0.728169
Epoch 238
Validation auc = 0.729108
Epoch 239
Loss = 2.8374e-01, PNorm = 44.3233, GNorm = 1.9163, lr_0 = 6.7200e-04
Validation auc = 0.728638
Epoch 240
Validation auc = 0.733333
Epoch 241
Validation auc = 0.736620
Epoch 242
Validation auc = 0.743662
Epoch 243
Validation auc = 0.740376
Epoch 244
Loss = 2.6964e-01, PNorm = 44.4131, GNorm = 1.9107, lr_0 = 6.6625e-04
Validation auc = 0.740376
Epoch 245
Validation auc = 0.746479
Epoch 246
Validation auc = 0.750704
Epoch 247
Validation auc = 0.750235
Epoch 248
Validation auc = 0.746948
Epoch 249
Loss = 3.0184e-01, PNorm = 44.4966, GNorm = 2.6694, lr_0 = 6.6056e-04
Validation auc = 0.739906
Epoch 250
Validation auc = 0.735681
Epoch 251
Validation auc = 0.733333
Epoch 252
Validation auc = 0.731455
Epoch 253
Validation auc = 0.730516
Epoch 254
Loss = 2.6537e-01, PNorm = 44.5765, GNorm = 2.5545, lr_0 = 6.5491e-04
Validation auc = 0.730047
Epoch 255
Validation auc = 0.731455
Epoch 256
Validation auc = 0.732394
Epoch 257
Validation auc = 0.733803
Epoch 258
Validation auc = 0.734272
Epoch 259
Loss = 2.5517e-01, PNorm = 44.6631, GNorm = 1.3606, lr_0 = 6.4931e-04
Validation auc = 0.735681
Epoch 260
Validation auc = 0.733333
Epoch 261
Validation auc = 0.729108
Epoch 262
Validation auc = 0.723474
Epoch 263
Validation auc = 0.717840
Epoch 264
Loss = 3.0369e-01, PNorm = 44.7385, GNorm = 2.0090, lr_0 = 6.4376e-04
Validation auc = 0.715962
Epoch 265
Validation auc = 0.714085
Epoch 266
Validation auc = 0.713615
Epoch 267
Validation auc = 0.715493
Epoch 268
Validation auc = 0.717840
Epoch 269
Loss = 2.3530e-01, PNorm = 44.8324, GNorm = 2.5031, lr_0 = 6.3826e-04
Validation auc = 0.717840
Epoch 270
Validation auc = 0.721596
Epoch 271
Validation auc = 0.722535
Epoch 272
Validation auc = 0.724413
Epoch 273
Validation auc = 0.726761
Epoch 274
Loss = 1.9287e-01, PNorm = 44.9034, GNorm = 1.1133, lr_0 = 6.3280e-04
Validation auc = 0.728638
Epoch 275
Validation auc = 0.726761
Epoch 276
Validation auc = 0.722535
Epoch 277
Validation auc = 0.720188
Epoch 278
Validation auc = 0.723944
Epoch 279
Loss = 2.6623e-01, PNorm = 44.9745, GNorm = 3.6322, lr_0 = 6.2739e-04
Validation auc = 0.723005
Epoch 280
Validation auc = 0.719249
Epoch 281
Validation auc = 0.715962
Epoch 282
Validation auc = 0.713146
Epoch 283
Validation auc = 0.715962
Epoch 284
Loss = 2.0346e-01, PNorm = 45.0538, GNorm = 3.4675, lr_0 = 6.2203e-04
Validation auc = 0.715493
Epoch 285
Validation auc = 0.723005
Epoch 286
Validation auc = 0.728169
Epoch 287
Validation auc = 0.721127
Epoch 288
Validation auc = 0.717371
Epoch 289
Loss = 2.7355e-01, PNorm = 45.1216, GNorm = 3.2732, lr_0 = 6.1671e-04
Validation auc = 0.713146
Epoch 290
Validation auc = 0.714085
Epoch 291
Validation auc = 0.707042
Epoch 292
Validation auc = 0.712207
Epoch 293
Validation auc = 0.710329
Epoch 294
Loss = 2.7110e-01, PNorm = 45.2007, GNorm = 4.1903, lr_0 = 6.1144e-04
Validation auc = 0.708451
Epoch 295
Validation auc = 0.709859
Epoch 296
Validation auc = 0.720657
Epoch 297
Validation auc = 0.723474
Epoch 298
Validation auc = 0.725352
Epoch 299
Loss = 2.2368e-01, PNorm = 45.3015, GNorm = 1.8506, lr_0 = 6.0621e-04
Validation auc = 0.722535
Model 0 best validation auc = 0.762911 on epoch 28
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.856111
Ensemble test auc = 0.856111
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.649765
Epoch 1
Validation auc = 0.674178
Epoch 2
Validation auc = 0.667606
Epoch 3
Validation auc = 0.668545
Epoch 4
Loss = 1.0876e+00, PNorm = 40.6664, GNorm = 15.9575, lr_0 = 6.5000e-04
Validation auc = 0.664789
Epoch 5
Validation auc = 0.671362
Epoch 6
Validation auc = 0.660563
Epoch 7
Validation auc = 0.659155
Epoch 8
Validation auc = 0.669484
Epoch 9
Loss = 1.0050e+00, PNorm = 40.7127, GNorm = 9.4112, lr_0 = 9.9743e-04
Validation auc = 0.670423
Epoch 10
Validation auc = 0.734742
Epoch 11
Validation auc = 0.642723
Epoch 12
Validation auc = 0.659624
Epoch 13
Validation auc = 0.672300
Epoch 14
Loss = 7.5446e-01, PNorm = 40.7584, GNorm = 7.9101, lr_0 = 9.8890e-04
Validation auc = 0.732394
Epoch 15
Validation auc = 0.777934
Epoch 16
Validation auc = 0.690610
Epoch 17
Validation auc = 0.690610
Epoch 18
Validation auc = 0.705634
Epoch 19
Loss = 6.8242e-01, PNorm = 40.7987, GNorm = 6.0677, lr_0 = 9.8045e-04
Validation auc = 0.683568
Epoch 20
Validation auc = 0.663380
Epoch 21
Validation auc = 0.658685
Epoch 22
Validation auc = 0.661972
Epoch 23
Validation auc = 0.666197
Epoch 24
Loss = 6.5581e-01, PNorm = 40.8466, GNorm = 4.5116, lr_0 = 9.7207e-04
Validation auc = 0.665258
Epoch 25
Validation auc = 0.664789
Epoch 26
Validation auc = 0.668545
Epoch 27
Validation auc = 0.682160
Epoch 28
Validation auc = 0.699061
Epoch 29
Loss = 5.6465e-01, PNorm = 40.8680, GNorm = 4.2024, lr_0 = 9.6376e-04
Validation auc = 0.712676
Epoch 30
Validation auc = 0.706573
Epoch 31
Validation auc = 0.697653
Epoch 32
Validation auc = 0.686385
Epoch 33
Validation auc = 0.666667
Epoch 34
Loss = 5.9160e-01, PNorm = 40.9107, GNorm = 9.3277, lr_0 = 9.5552e-04
Validation auc = 0.640845
Epoch 35
Validation auc = 0.631455
Epoch 36
Validation auc = 0.620657
Epoch 37
Validation auc = 0.632394
Epoch 38
Validation auc = 0.625822
Epoch 39
Loss = 6.4413e-01, PNorm = 40.9579, GNorm = 4.8620, lr_0 = 9.4735e-04
Validation auc = 0.621127
Epoch 40
Validation auc = 0.617371
Epoch 41
Validation auc = 0.618310
Epoch 42
Validation auc = 0.614554
Epoch 43
Validation auc = 0.613146
Epoch 44
Loss = 5.8825e-01, PNorm = 40.9844, GNorm = 3.8137, lr_0 = 9.3925e-04
Validation auc = 0.614085
Epoch 45
Validation auc = 0.631455
Epoch 46
Validation auc = 0.635211
Epoch 47
Validation auc = 0.625822
Epoch 48
Validation auc = 0.632864
Epoch 49
Loss = 5.8276e-01, PNorm = 41.0387, GNorm = 4.7174, lr_0 = 9.3122e-04
Validation auc = 0.635681
Epoch 50
Validation auc = 0.620657
Epoch 51
Validation auc = 0.626761
Epoch 52
Validation auc = 0.658685
Epoch 53
Validation auc = 0.652582
Epoch 54
Loss = 5.7388e-01, PNorm = 41.0799, GNorm = 2.5862, lr_0 = 9.2326e-04
Validation auc = 0.643192
Epoch 55
Validation auc = 0.643192
Epoch 56
Validation auc = 0.639437
Epoch 57
Validation auc = 0.649765
Epoch 58
Validation auc = 0.672770
Epoch 59
Loss = 5.8007e-01, PNorm = 41.1300, GNorm = 5.0977, lr_0 = 9.1537e-04
Validation auc = 0.650704
Epoch 60
Validation auc = 0.637559
Epoch 61
Validation auc = 0.661502
Epoch 62
Validation auc = 0.662911
Epoch 63
Validation auc = 0.657746
Epoch 64
Loss = 5.7533e-01, PNorm = 41.1737, GNorm = 4.6470, lr_0 = 9.0754e-04
Validation auc = 0.653052
Epoch 65
Validation auc = 0.655399
Epoch 66
Validation auc = 0.668075
Epoch 67
Validation auc = 0.665258
Epoch 68
Validation auc = 0.645540
Epoch 69
Loss = 5.4912e-01, PNorm = 41.2251, GNorm = 3.4596, lr_0 = 8.9978e-04
Validation auc = 0.631925
Epoch 70
Validation auc = 0.637089
Epoch 71
Validation auc = 0.642723
Epoch 72
Validation auc = 0.640376
Epoch 73
Validation auc = 0.643192
Epoch 74
Loss = 5.6889e-01, PNorm = 41.2884, GNorm = 3.3854, lr_0 = 8.9209e-04
Validation auc = 0.638028
Epoch 75
Validation auc = 0.638967
Epoch 76
Validation auc = 0.634742
Epoch 77
Validation auc = 0.636150
Epoch 78
Validation auc = 0.645070
Epoch 79
Loss = 5.4145e-01, PNorm = 41.3278, GNorm = 2.9870, lr_0 = 8.8447e-04
Validation auc = 0.645070
Epoch 80
Validation auc = 0.652582
Epoch 81
Validation auc = 0.648826
Epoch 82
Validation auc = 0.640376
Epoch 83
Validation auc = 0.635211
Epoch 84
Loss = 4.8474e-01, PNorm = 41.3881, GNorm = 3.2753, lr_0 = 8.7691e-04
Validation auc = 0.644131
Epoch 85
Validation auc = 0.643662
Epoch 86
Validation auc = 0.645540
Epoch 87
Validation auc = 0.653991
Epoch 88
Validation auc = 0.647887
Epoch 89
Loss = 4.7768e-01, PNorm = 41.4428, GNorm = 2.8218, lr_0 = 8.6941e-04
Validation auc = 0.647418
Epoch 90
Validation auc = 0.646948
Epoch 91
Validation auc = 0.664789
Epoch 92
Validation auc = 0.700939
Epoch 93
Validation auc = 0.701878
Epoch 94
Loss = 5.0753e-01, PNorm = 41.5239, GNorm = 2.3757, lr_0 = 8.6198e-04
Validation auc = 0.696244
Epoch 95
Validation auc = 0.680751
Epoch 96
Validation auc = 0.669484
Epoch 97
Validation auc = 0.661972
Epoch 98
Validation auc = 0.671362
Epoch 99
Loss = 4.0298e-01, PNorm = 41.6022, GNorm = 2.5264, lr_0 = 8.5461e-04
Validation auc = 0.672770
Epoch 100
Validation auc = 0.675117
Epoch 101
Validation auc = 0.675117
Epoch 102
Validation auc = 0.666667
Epoch 103
Validation auc = 0.662441
Epoch 104
Loss = 5.3391e-01, PNorm = 41.6816, GNorm = 3.8458, lr_0 = 8.4730e-04
Validation auc = 0.659624
Epoch 105
Validation auc = 0.661502
Epoch 106
Validation auc = 0.662911
Epoch 107
Validation auc = 0.670892
Epoch 108
Validation auc = 0.684038
Epoch 109
Loss = 4.4263e-01, PNorm = 41.7511, GNorm = 2.5768, lr_0 = 8.4006e-04
Validation auc = 0.695305
Epoch 110
Validation auc = 0.704695
Epoch 111
Validation auc = 0.707512
Epoch 112
Validation auc = 0.708451
Epoch 113
Validation auc = 0.706573
Epoch 114
Loss = 4.4434e-01, PNorm = 41.8346, GNorm = 2.3888, lr_0 = 8.3288e-04
Validation auc = 0.700939
Epoch 115
Validation auc = 0.692488
Epoch 116
Validation auc = 0.686854
Epoch 117
Validation auc = 0.686854
Epoch 118
Validation auc = 0.696714
Epoch 119
Loss = 4.9984e-01, PNorm = 41.9289, GNorm = 2.5198, lr_0 = 8.2576e-04
Validation auc = 0.712676
Epoch 120
Validation auc = 0.728169
Epoch 121
Validation auc = 0.732394
Epoch 122
Validation auc = 0.727700
Epoch 123
Validation auc = 0.721596
Epoch 124
Loss = 4.4344e-01, PNorm = 42.0328, GNorm = 2.4286, lr_0 = 8.1870e-04
Validation auc = 0.711737
Epoch 125
Validation auc = 0.712207
Epoch 126
Validation auc = 0.710798
Epoch 127
Validation auc = 0.698122
Epoch 128
Validation auc = 0.702817
Epoch 129
Loss = 5.1110e-01, PNorm = 42.1331, GNorm = 3.1030, lr_0 = 8.1170e-04
Validation auc = 0.705164
Epoch 130
Validation auc = 0.704225
Epoch 131
Validation auc = 0.712207
Epoch 132
Validation auc = 0.719249
Epoch 133
Validation auc = 0.717371
Epoch 134
Loss = 4.0428e-01, PNorm = 42.2440, GNorm = 2.0037, lr_0 = 8.0476e-04
Validation auc = 0.722066
Epoch 135
Validation auc = 0.720188
Epoch 136
Validation auc = 0.707981
Epoch 137
Validation auc = 0.702347
Epoch 138
Validation auc = 0.699531
Epoch 139
Loss = 3.4503e-01, PNorm = 42.3450, GNorm = 2.4817, lr_0 = 7.9788e-04
Validation auc = 0.700939
Epoch 140
Validation auc = 0.709390
Epoch 141
Validation auc = 0.715493
Epoch 142
Validation auc = 0.716432
Epoch 143
Validation auc = 0.721127
Epoch 144
Loss = 3.9864e-01, PNorm = 42.4499, GNorm = 2.7479, lr_0 = 7.9106e-04
Validation auc = 0.723474
Epoch 145
Validation auc = 0.729577
Epoch 146
Validation auc = 0.737089
Epoch 147
Validation auc = 0.746479
Epoch 148
Validation auc = 0.747418
Epoch 149
Loss = 4.3380e-01, PNorm = 42.5534, GNorm = 3.7717, lr_0 = 7.8430e-04
Validation auc = 0.738028
Epoch 150
Validation auc = 0.744601
Epoch 151
Validation auc = 0.737559
Epoch 152
Validation auc = 0.726761
Epoch 153
Validation auc = 0.733333
Epoch 154
Loss = 3.4926e-01, PNorm = 42.6534, GNorm = 2.7665, lr_0 = 7.7759e-04
Validation auc = 0.732394
Epoch 155
Validation auc = 0.748357
Epoch 156
Validation auc = 0.756338
Epoch 157
Validation auc = 0.756808
Epoch 158
Validation auc = 0.751643
Epoch 159
Loss = 3.6341e-01, PNorm = 42.7375, GNorm = 2.2139, lr_0 = 7.7095e-04
Validation auc = 0.746479
Epoch 160
Validation auc = 0.755869
Epoch 161
Validation auc = 0.760563
Epoch 162
Validation auc = 0.755399
Epoch 163
Validation auc = 0.758685
Epoch 164
Loss = 3.8049e-01, PNorm = 42.8171, GNorm = 2.4252, lr_0 = 7.6436e-04
Validation auc = 0.755399
Epoch 165
Validation auc = 0.749765
Epoch 166
Validation auc = 0.748826
Epoch 167
Validation auc = 0.746009
Epoch 168
Validation auc = 0.724883
Epoch 169
Loss = 3.8821e-01, PNorm = 42.9019, GNorm = 1.6559, lr_0 = 7.5782e-04
Validation auc = 0.715493
Epoch 170
Validation auc = 0.717371
Epoch 171
Validation auc = 0.715023
Epoch 172
Validation auc = 0.717371
Epoch 173
Validation auc = 0.720657
Epoch 174
Loss = 3.8514e-01, PNorm = 43.0044, GNorm = 2.9102, lr_0 = 7.5134e-04
Validation auc = 0.724883
Epoch 175
Validation auc = 0.730047
Epoch 176
Validation auc = 0.730986
Epoch 177
Validation auc = 0.729577
Epoch 178
Validation auc = 0.734272
Epoch 179
Loss = 3.1922e-01, PNorm = 43.0983, GNorm = 2.4689, lr_0 = 7.4492e-04
Validation auc = 0.730986
Epoch 180
Validation auc = 0.730047
Epoch 181
Validation auc = 0.721596
Epoch 182
Validation auc = 0.735681
Epoch 183
Validation auc = 0.750235
Epoch 184
Loss = 3.7262e-01, PNorm = 43.1781, GNorm = 2.8077, lr_0 = 7.3855e-04
Validation auc = 0.761033
Epoch 185
Validation auc = 0.761033
Epoch 186
Validation auc = 0.758685
Epoch 187
Validation auc = 0.756338
Epoch 188
Validation auc = 0.750704
Epoch 189
Loss = 5.2751e-01, PNorm = 43.2531, GNorm = 4.5642, lr_0 = 7.3224e-04
Validation auc = 0.749296
Epoch 190
Validation auc = 0.746479
Epoch 191
Validation auc = 0.749296
Epoch 192
Validation auc = 0.744601
Epoch 193
Validation auc = 0.733803
Epoch 194
Loss = 3.9580e-01, PNorm = 43.3283, GNorm = 3.3242, lr_0 = 7.2598e-04
Validation auc = 0.738028
Epoch 195
Validation auc = 0.744131
Epoch 196
Validation auc = 0.753521
Epoch 197
Validation auc = 0.756808
Epoch 198
Validation auc = 0.756808
Epoch 199
Loss = 3.0891e-01, PNorm = 43.4174, GNorm = 1.9664, lr_0 = 7.1977e-04
Validation auc = 0.760094
Epoch 200
Validation auc = 0.755869
Epoch 201
Validation auc = 0.753052
Epoch 202
Validation auc = 0.753991
Epoch 203
Validation auc = 0.750704
Epoch 204
Loss = 2.8270e-01, PNorm = 43.5025, GNorm = 2.6705, lr_0 = 7.1362e-04
Validation auc = 0.747418
Epoch 205
Validation auc = 0.741315
Epoch 206
Validation auc = 0.751174
Epoch 207
Validation auc = 0.744131
Epoch 208
Validation auc = 0.739906
Epoch 209
Loss = 3.6347e-01, PNorm = 43.5936, GNorm = 3.9375, lr_0 = 7.0752e-04
Validation auc = 0.738028
Epoch 210
Validation auc = 0.733333
Epoch 211
Validation auc = 0.735681
Epoch 212
Validation auc = 0.742723
Epoch 213
Validation auc = 0.750704
Epoch 214
Loss = 2.4105e-01, PNorm = 43.6841, GNorm = 1.9146, lr_0 = 7.0147e-04
Validation auc = 0.756338
Epoch 215
Validation auc = 0.760563
Epoch 216
Validation auc = 0.763380
Epoch 217
Validation auc = 0.761033
Epoch 218
Validation auc = 0.754460
Epoch 219
Loss = 3.0605e-01, PNorm = 43.7635, GNorm = 2.8301, lr_0 = 6.9548e-04
Validation auc = 0.753052
Epoch 220
Validation auc = 0.754460
Epoch 221
Validation auc = 0.746479
Epoch 222
Validation auc = 0.746479
Epoch 223
Validation auc = 0.740845
Epoch 224
Loss = 3.8233e-01, PNorm = 43.8460, GNorm = 3.8259, lr_0 = 6.8953e-04
Validation auc = 0.733803
Epoch 225
Validation auc = 0.718779
Epoch 226
Validation auc = 0.718310
Epoch 227
Validation auc = 0.712676
Epoch 228
Validation auc = 0.713146
Epoch 229
Loss = 2.6902e-01, PNorm = 43.9145, GNorm = 2.7136, lr_0 = 6.8364e-04
Validation auc = 0.713615
Epoch 230
Validation auc = 0.711737
Epoch 231
Validation auc = 0.718779
Epoch 232
Validation auc = 0.728638
Epoch 233
Validation auc = 0.737559
Epoch 234
Loss = 2.9793e-01, PNorm = 44.0015, GNorm = 2.6566, lr_0 = 6.7779e-04
Validation auc = 0.741784
Epoch 235
Validation auc = 0.741784
Epoch 236
Validation auc = 0.742723
Epoch 237
Validation auc = 0.740845
Epoch 238
Validation auc = 0.748826
Epoch 239
Loss = 2.9457e-01, PNorm = 44.0980, GNorm = 2.9897, lr_0 = 6.7200e-04
Validation auc = 0.755399
Epoch 240
Validation auc = 0.752582
Epoch 241
Validation auc = 0.738498
Epoch 242
Validation auc = 0.732864
Epoch 243
Validation auc = 0.728638
Epoch 244
Loss = 3.8543e-01, PNorm = 44.1730, GNorm = 4.5694, lr_0 = 6.6625e-04
Validation auc = 0.726761
Epoch 245
Validation auc = 0.733333
Epoch 246
Validation auc = 0.733333
Epoch 247
Validation auc = 0.738967
Epoch 248
Validation auc = 0.743662
Epoch 249
Loss = 3.9622e-01, PNorm = 44.2576, GNorm = 7.8824, lr_0 = 6.6056e-04
Validation auc = 0.742723
Epoch 250
Validation auc = 0.731455
Epoch 251
Validation auc = 0.715023
Epoch 252
Validation auc = 0.709859
Epoch 253
Validation auc = 0.714085
Epoch 254
Loss = 3.5730e-01, PNorm = 44.3315, GNorm = 2.6143, lr_0 = 6.5491e-04
Validation auc = 0.714085
Epoch 255
Validation auc = 0.715493
Epoch 256
Validation auc = 0.716432
Epoch 257
Validation auc = 0.721596
Epoch 258
Validation auc = 0.720188
Epoch 259
Loss = 3.4132e-01, PNorm = 44.3949, GNorm = 5.0619, lr_0 = 6.4931e-04
Validation auc = 0.722066
Epoch 260
Validation auc = 0.730047
Epoch 261
Validation auc = 0.730047
Epoch 262
Validation auc = 0.732394
Epoch 263
Validation auc = 0.729108
Epoch 264
Loss = 3.3519e-01, PNorm = 44.4651, GNorm = 4.9413, lr_0 = 6.4376e-04
Validation auc = 0.730986
Epoch 265
Validation auc = 0.733333
Epoch 266
Validation auc = 0.729108
Epoch 267
Validation auc = 0.729577
Epoch 268
Validation auc = 0.725822
Epoch 269
Loss = 2.8320e-01, PNorm = 44.5310, GNorm = 1.9137, lr_0 = 6.3826e-04
Validation auc = 0.730047
Epoch 270
Validation auc = 0.729577
Epoch 271
Validation auc = 0.727230
Epoch 272
Validation auc = 0.735681
Epoch 273
Validation auc = 0.741315
Epoch 274
Loss = 2.5834e-01, PNorm = 44.5905, GNorm = 2.5606, lr_0 = 6.3280e-04
Validation auc = 0.741315
Epoch 275
Validation auc = 0.737089
Epoch 276
Validation auc = 0.736620
Epoch 277
Validation auc = 0.730986
Epoch 278
Validation auc = 0.732394
Epoch 279
Loss = 2.2140e-01, PNorm = 44.6544, GNorm = 1.7767, lr_0 = 6.2739e-04
Validation auc = 0.731925
Epoch 280
Validation auc = 0.734272
Epoch 281
Validation auc = 0.734272
Epoch 282
Validation auc = 0.731925
Epoch 283
Validation auc = 0.731925
Epoch 284
Loss = 2.2407e-01, PNorm = 44.7271, GNorm = 2.9690, lr_0 = 6.2203e-04
Validation auc = 0.734272
Epoch 285
Validation auc = 0.736150
Epoch 286
Validation auc = 0.737089
Epoch 287
Validation auc = 0.743192
Epoch 288
Validation auc = 0.737559
Epoch 289
Loss = 2.5983e-01, PNorm = 44.7923, GNorm = 2.3640, lr_0 = 6.1671e-04
Validation auc = 0.734742
Epoch 290
Validation auc = 0.731455
Epoch 291
Validation auc = 0.726761
Epoch 292
Validation auc = 0.721596
Epoch 293
Validation auc = 0.718779
Epoch 294
Loss = 2.5323e-01, PNorm = 44.8473, GNorm = 2.1431, lr_0 = 6.1144e-04
Validation auc = 0.718310
Epoch 295
Validation auc = 0.724413
Epoch 296
Validation auc = 0.726761
Epoch 297
Validation auc = 0.730047
Epoch 298
Validation auc = 0.735211
Epoch 299
Loss = 2.3729e-01, PNorm = 44.9183, GNorm = 2.0203, lr_0 = 6.0621e-04
Validation auc = 0.738967
Model 0 best validation auc = 0.777934 on epoch 15
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.779049
Ensemble test auc = 0.779049
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.661972
Epoch 1
Validation auc = 0.682160
Epoch 2
Validation auc = 0.680751
Epoch 3
Validation auc = 0.684507
Epoch 4
Loss = 1.1781e+00, PNorm = 40.6682, GNorm = 15.0723, lr_0 = 6.5000e-04
Validation auc = 0.689671
Epoch 5
Validation auc = 0.723474
Epoch 6
Validation auc = 0.690610
Epoch 7
Validation auc = 0.663380
Epoch 8
Validation auc = 0.659624
Epoch 9
Loss = 9.1711e-01, PNorm = 40.7127, GNorm = 7.3795, lr_0 = 9.9743e-04
Validation auc = 0.662911
Epoch 10
Validation auc = 0.651174
Epoch 11
Validation auc = 0.634742
Epoch 12
Validation auc = 0.628638
Epoch 13
Validation auc = 0.628169
Epoch 14
Loss = 7.2090e-01, PNorm = 40.7645, GNorm = 9.7656, lr_0 = 9.8890e-04
Validation auc = 0.651174
Epoch 15
Validation auc = 0.661502
Epoch 16
Validation auc = 0.652113
Epoch 17
Validation auc = 0.659624
Epoch 18
Validation auc = 0.669953
Epoch 19
Loss = 8.5540e-01, PNorm = 40.8080, GNorm = 7.9993, lr_0 = 9.8045e-04
Validation auc = 0.637559
Epoch 20
Validation auc = 0.624883
Epoch 21
Validation auc = 0.628169
Epoch 22
Validation auc = 0.634272
Epoch 23
Validation auc = 0.636150
Epoch 24
Loss = 6.2496e-01, PNorm = 40.8435, GNorm = 6.3456, lr_0 = 9.7207e-04
Validation auc = 0.643662
Epoch 25
Validation auc = 0.647887
Epoch 26
Validation auc = 0.653052
Epoch 27
Validation auc = 0.651643
Epoch 28
Validation auc = 0.644601
Epoch 29
Loss = 7.7754e-01, PNorm = 40.8808, GNorm = 5.8928, lr_0 = 9.6376e-04
Validation auc = 0.630986
Epoch 30
Validation auc = 0.643662
Epoch 31
Validation auc = 0.648357
Epoch 32
Validation auc = 0.630986
Epoch 33
Validation auc = 0.633803
Epoch 34
Loss = 6.5506e-01, PNorm = 40.9031, GNorm = 3.4445, lr_0 = 9.5552e-04
Validation auc = 0.630986
Epoch 35
Validation auc = 0.641315
Epoch 36
Validation auc = 0.641315
Epoch 37
Validation auc = 0.643192
Epoch 38
Validation auc = 0.650235
Epoch 39
Loss = 6.6351e-01, PNorm = 40.9500, GNorm = 5.1138, lr_0 = 9.4735e-04
Validation auc = 0.656338
Epoch 40
Validation auc = 0.653991
Epoch 41
Validation auc = 0.645540
Epoch 42
Validation auc = 0.643192
Epoch 43
Validation auc = 0.648826
Epoch 44
Loss = 5.9955e-01, PNorm = 40.9732, GNorm = 4.3191, lr_0 = 9.3925e-04
Validation auc = 0.638498
Epoch 45
Validation auc = 0.631925
Epoch 46
Validation auc = 0.623005
Epoch 47
Validation auc = 0.628169
Epoch 48
Validation auc = 0.626761
Epoch 49
Loss = 6.6602e-01, PNorm = 41.0217, GNorm = 3.8606, lr_0 = 9.3122e-04
Validation auc = 0.620657
Epoch 50
Validation auc = 0.616432
Epoch 51
Validation auc = 0.622066
Epoch 52
Validation auc = 0.633803
Epoch 53
Validation auc = 0.640376
Epoch 54
Loss = 5.4955e-01, PNorm = 41.0629, GNorm = 4.5238, lr_0 = 9.2326e-04
Validation auc = 0.633803
Epoch 55
Validation auc = 0.647887
Epoch 56
Validation auc = 0.655869
Epoch 57
Validation auc = 0.667136
Epoch 58
Validation auc = 0.675117
Epoch 59
Loss = 5.9369e-01, PNorm = 41.1049, GNorm = 4.1474, lr_0 = 9.1537e-04
Validation auc = 0.671831
Epoch 60
Validation auc = 0.663850
Epoch 61
Validation auc = 0.662441
Epoch 62
Validation auc = 0.657746
Epoch 63
Validation auc = 0.648357
Epoch 64
Loss = 5.3629e-01, PNorm = 41.1613, GNorm = 3.8278, lr_0 = 9.0754e-04
Validation auc = 0.640376
Epoch 65
Validation auc = 0.648357
Epoch 66
Validation auc = 0.652113
Epoch 67
Validation auc = 0.651174
Epoch 68
Validation auc = 0.649296
Epoch 69
Loss = 5.0683e-01, PNorm = 41.2254, GNorm = 2.8638, lr_0 = 8.9978e-04
Validation auc = 0.648357
Epoch 70
Validation auc = 0.641784
Epoch 71
Validation auc = 0.638498
Epoch 72
Validation auc = 0.629577
Epoch 73
Validation auc = 0.630516
Epoch 74
Loss = 5.5573e-01, PNorm = 41.2940, GNorm = 3.4333, lr_0 = 8.9209e-04
Validation auc = 0.637089
Epoch 75
Validation auc = 0.648357
Epoch 76
Validation auc = 0.658685
Epoch 77
Validation auc = 0.659624
Epoch 78
Validation auc = 0.653991
Epoch 79
Loss = 5.9418e-01, PNorm = 41.3519, GNorm = 2.8465, lr_0 = 8.8447e-04
Validation auc = 0.635211
Epoch 80
Validation auc = 0.623474
Epoch 81
Validation auc = 0.617840
Epoch 82
Validation auc = 0.624883
Epoch 83
Validation auc = 0.633803
Epoch 84
Loss = 4.4913e-01, PNorm = 41.4261, GNorm = 2.9812, lr_0 = 8.7691e-04
Validation auc = 0.644131
Epoch 85
Validation auc = 0.644601
Epoch 86
Validation auc = 0.630986
Epoch 87
Validation auc = 0.631455
Epoch 88
Validation auc = 0.634272
Epoch 89
Loss = 5.2753e-01, PNorm = 41.4811, GNorm = 3.4263, lr_0 = 8.6941e-04
Validation auc = 0.638028
Epoch 90
Validation auc = 0.640376
Epoch 91
Validation auc = 0.637089
Epoch 92
Validation auc = 0.642723
Epoch 93
Validation auc = 0.645070
Epoch 94
Loss = 4.1392e-01, PNorm = 41.5556, GNorm = 2.8698, lr_0 = 8.6198e-04
Validation auc = 0.648826
Epoch 95
Validation auc = 0.652582
Epoch 96
Validation auc = 0.653521
Epoch 97
Validation auc = 0.659155
Epoch 98
Validation auc = 0.654930
Epoch 99
Loss = 4.4405e-01, PNorm = 41.6218, GNorm = 3.5428, lr_0 = 8.5461e-04
Validation auc = 0.658216
Epoch 100
Validation auc = 0.647887
Epoch 101
Validation auc = 0.644601
Epoch 102
Validation auc = 0.646948
Epoch 103
Validation auc = 0.650704
Epoch 104
Loss = 5.5670e-01, PNorm = 41.6764, GNorm = 3.3436, lr_0 = 8.4730e-04
Validation auc = 0.645540
Epoch 105
Validation auc = 0.647887
Epoch 106
Validation auc = 0.657277
Epoch 107
Validation auc = 0.664319
Epoch 108
Validation auc = 0.675587
Epoch 109
Loss = 4.8457e-01, PNorm = 41.7572, GNorm = 3.0200, lr_0 = 8.4006e-04
Validation auc = 0.669014
Epoch 110
Validation auc = 0.661033
Epoch 111
Validation auc = 0.668075
Epoch 112
Validation auc = 0.671362
Epoch 113
Validation auc = 0.676056
Epoch 114
Loss = 4.9363e-01, PNorm = 41.8322, GNorm = 2.5894, lr_0 = 8.3288e-04
Validation auc = 0.684038
Epoch 115
Validation auc = 0.688732
Epoch 116
Validation auc = 0.680282
Epoch 117
Validation auc = 0.673709
Epoch 118
Validation auc = 0.661033
Epoch 119
Loss = 5.2572e-01, PNorm = 41.9128, GNorm = 2.5399, lr_0 = 8.2576e-04
Validation auc = 0.661502
Epoch 120
Validation auc = 0.662441
Epoch 121
Validation auc = 0.666197
Epoch 122
Validation auc = 0.669484
Epoch 123
Validation auc = 0.660094
Epoch 124
Loss = 4.9607e-01, PNorm = 42.0133, GNorm = 2.9986, lr_0 = 8.1870e-04
Validation auc = 0.657746
Epoch 125
Validation auc = 0.661972
Epoch 126
Validation auc = 0.678404
Epoch 127
Validation auc = 0.670423
Epoch 128
Validation auc = 0.661033
Epoch 129
Loss = 3.9704e-01, PNorm = 42.1044, GNorm = 2.5767, lr_0 = 8.1170e-04
Validation auc = 0.659155
Epoch 130
Validation auc = 0.662911
Epoch 131
Validation auc = 0.667136
Epoch 132
Validation auc = 0.672300
Epoch 133
Validation auc = 0.683099
Epoch 134
Loss = 4.0641e-01, PNorm = 42.2112, GNorm = 2.7935, lr_0 = 8.0476e-04
Validation auc = 0.700000
Epoch 135
Validation auc = 0.689671
Epoch 136
Validation auc = 0.687324
Epoch 137
Validation auc = 0.682629
Epoch 138
Validation auc = 0.678873
Epoch 139
Loss = 4.5826e-01, PNorm = 42.2806, GNorm = 4.3042, lr_0 = 7.9788e-04
Validation auc = 0.684977
Epoch 140
Validation auc = 0.693427
Epoch 141
Validation auc = 0.693897
Epoch 142
Validation auc = 0.694366
Epoch 143
Validation auc = 0.696244
Epoch 144
Loss = 4.8512e-01, PNorm = 42.3697, GNorm = 3.2741, lr_0 = 7.9106e-04
Validation auc = 0.704695
Epoch 145
Validation auc = 0.705634
Epoch 146
Validation auc = 0.713615
Epoch 147
Validation auc = 0.714554
Epoch 148
Validation auc = 0.716432
Epoch 149
Loss = 4.7121e-01, PNorm = 42.4670, GNorm = 4.2043, lr_0 = 7.8430e-04
Validation auc = 0.710798
Epoch 150
Validation auc = 0.705634
Epoch 151
Validation auc = 0.695775
Epoch 152
Validation auc = 0.694366
Epoch 153
Validation auc = 0.704695
Epoch 154
Loss = 3.6228e-01, PNorm = 42.5743, GNorm = 2.4236, lr_0 = 7.7759e-04
Validation auc = 0.704695
Epoch 155
Validation auc = 0.715493
Epoch 156
Validation auc = 0.724413
Epoch 157
Validation auc = 0.724883
Epoch 158
Validation auc = 0.724883
Epoch 159
Loss = 5.3620e-01, PNorm = 42.6870, GNorm = 4.2912, lr_0 = 7.7095e-04
Validation auc = 0.724413
Epoch 160
Validation auc = 0.727700
Epoch 161
Validation auc = 0.720188
Epoch 162
Validation auc = 0.713146
Epoch 163
Validation auc = 0.709390
Epoch 164
Loss = 3.9207e-01, PNorm = 42.7686, GNorm = 3.4823, lr_0 = 7.6436e-04
Validation auc = 0.710329
Epoch 165
Validation auc = 0.708451
Epoch 166
Validation auc = 0.709390
Epoch 167
Validation auc = 0.702347
Epoch 168
Validation auc = 0.702817
Epoch 169
Loss = 3.3011e-01, PNorm = 42.8708, GNorm = 2.2544, lr_0 = 7.5782e-04
Validation auc = 0.705164
Epoch 170
Validation auc = 0.704695
Epoch 171
Validation auc = 0.702817
Epoch 172
Validation auc = 0.707512
Epoch 173
Validation auc = 0.712676
Epoch 174
Loss = 3.4080e-01, PNorm = 42.9855, GNorm = 2.7074, lr_0 = 7.5134e-04
Validation auc = 0.709390
Epoch 175
Validation auc = 0.710329
Epoch 176
Validation auc = 0.713146
Epoch 177
Validation auc = 0.714554
Epoch 178
Validation auc = 0.715493
Epoch 179
Loss = 3.6012e-01, PNorm = 43.0959, GNorm = 2.5332, lr_0 = 7.4492e-04
Validation auc = 0.713615
Epoch 180
Validation auc = 0.719249
Epoch 181
Validation auc = 0.723474
Epoch 182
Validation auc = 0.715962
Epoch 183
Validation auc = 0.715023
Epoch 184
Loss = 4.3139e-01, PNorm = 43.1846, GNorm = 3.0004, lr_0 = 7.3855e-04
Validation auc = 0.712207
Epoch 185
Validation auc = 0.708920
Epoch 186
Validation auc = 0.715493
Epoch 187
Validation auc = 0.718310
Epoch 188
Validation auc = 0.721596
Epoch 189
Loss = 4.0971e-01, PNorm = 43.2773, GNorm = 4.3169, lr_0 = 7.3224e-04
Validation auc = 0.722066
Epoch 190
Validation auc = 0.707981
Epoch 191
Validation auc = 0.704695
Epoch 192
Validation auc = 0.713146
Epoch 193
Validation auc = 0.715493
Epoch 194
Loss = 4.0989e-01, PNorm = 43.3738, GNorm = 3.6659, lr_0 = 7.2598e-04
Validation auc = 0.717371
Epoch 195
Validation auc = 0.721127
Epoch 196
Validation auc = 0.722535
Epoch 197
Validation auc = 0.723944
Epoch 198
Validation auc = 0.718779
Epoch 199
Loss = 3.4719e-01, PNorm = 43.4653, GNorm = 2.5299, lr_0 = 7.1977e-04
Validation auc = 0.720657
Epoch 200
Validation auc = 0.715023
Epoch 201
Validation auc = 0.710329
Epoch 202
Validation auc = 0.712676
Epoch 203
Validation auc = 0.712207
Epoch 204
Loss = 3.1557e-01, PNorm = 43.5613, GNorm = 3.0541, lr_0 = 7.1362e-04
Validation auc = 0.713146
Epoch 205
Validation auc = 0.715493
Epoch 206
Validation auc = 0.715962
Epoch 207
Validation auc = 0.719249
Epoch 208
Validation auc = 0.719249
Epoch 209
Loss = 3.7296e-01, PNorm = 43.6522, GNorm = 3.5219, lr_0 = 7.0752e-04
Validation auc = 0.717840
Epoch 210
Validation auc = 0.715962
Epoch 211
Validation auc = 0.720657
Epoch 212
Validation auc = 0.719718
Epoch 213
Validation auc = 0.720188
Epoch 214
Loss = 3.6599e-01, PNorm = 43.7607, GNorm = 2.3538, lr_0 = 7.0147e-04
Validation auc = 0.723474
Epoch 215
Validation auc = 0.727230
Epoch 216
Validation auc = 0.728169
Epoch 217
Validation auc = 0.725352
Epoch 218
Validation auc = 0.726761
Epoch 219
Loss = 3.6650e-01, PNorm = 43.8413, GNorm = 2.9062, lr_0 = 6.9548e-04
Validation auc = 0.729108
Epoch 220
Validation auc = 0.730047
Epoch 221
Validation auc = 0.732394
Epoch 222
Validation auc = 0.735211
Epoch 223
Validation auc = 0.727700
Epoch 224
Loss = 3.4001e-01, PNorm = 43.9360, GNorm = 3.8500, lr_0 = 6.8953e-04
Validation auc = 0.722066
Epoch 225
Validation auc = 0.712207
Epoch 226
Validation auc = 0.714554
Epoch 227
Validation auc = 0.716901
Epoch 228
Validation auc = 0.724883
Epoch 229
Loss = 3.3092e-01, PNorm = 44.0463, GNorm = 2.6432, lr_0 = 6.8364e-04
Validation auc = 0.723944
Epoch 230
Validation auc = 0.726291
Epoch 231
Validation auc = 0.726761
Epoch 232
Validation auc = 0.729108
Epoch 233
Validation auc = 0.731455
Epoch 234
Loss = 2.2783e-01, PNorm = 44.1461, GNorm = 1.5628, lr_0 = 6.7779e-04
Validation auc = 0.730986
Epoch 235
Validation auc = 0.729577
Epoch 236
Validation auc = 0.729108
Epoch 237
Validation auc = 0.733333
Epoch 238
Validation auc = 0.735681
Epoch 239
Loss = 3.5763e-01, PNorm = 44.2374, GNorm = 2.4709, lr_0 = 6.7200e-04
Validation auc = 0.747418
Epoch 240
Validation auc = 0.762911
Epoch 241
Validation auc = 0.755399
Epoch 242
Validation auc = 0.747887
Epoch 243
Validation auc = 0.745070
Epoch 244
Loss = 3.6791e-01, PNorm = 44.3307, GNorm = 3.2160, lr_0 = 6.6625e-04
Validation auc = 0.734272
Epoch 245
Validation auc = 0.734742
Epoch 246
Validation auc = 0.729577
Epoch 247
Validation auc = 0.730047
Epoch 248
Validation auc = 0.727230
Epoch 249
Loss = 2.8694e-01, PNorm = 44.4332, GNorm = 1.6074, lr_0 = 6.6056e-04
Validation auc = 0.728638
Epoch 250
Validation auc = 0.727230
Epoch 251
Validation auc = 0.724883
Epoch 252
Validation auc = 0.723944
Epoch 253
Validation auc = 0.721596
Epoch 254
Loss = 2.2079e-01, PNorm = 44.5318, GNorm = 1.7543, lr_0 = 6.5491e-04
Validation auc = 0.722535
Epoch 255
Validation auc = 0.724883
Epoch 256
Validation auc = 0.727230
Epoch 257
Validation auc = 0.728638
Epoch 258
Validation auc = 0.724413
Epoch 259
Loss = 2.3631e-01, PNorm = 44.6401, GNorm = 3.1860, lr_0 = 6.4931e-04
Validation auc = 0.721596
Epoch 260
Validation auc = 0.723005
Epoch 261
Validation auc = 0.723005
Epoch 262
Validation auc = 0.722066
Epoch 263
Validation auc = 0.721596
Epoch 264
Loss = 2.1751e-01, PNorm = 44.7279, GNorm = 2.6049, lr_0 = 6.4376e-04
Validation auc = 0.721127
Epoch 265
Validation auc = 0.723005
Epoch 266
Validation auc = 0.724413
Epoch 267
Validation auc = 0.720657
Epoch 268
Validation auc = 0.720188
Epoch 269
Loss = 2.7057e-01, PNorm = 44.8017, GNorm = 2.4837, lr_0 = 6.3826e-04
Validation auc = 0.715023
Epoch 270
Validation auc = 0.709390
Epoch 271
Validation auc = 0.710329
Epoch 272
Validation auc = 0.707981
Epoch 273
Validation auc = 0.710329
Epoch 274
Loss = 2.6608e-01, PNorm = 44.8758, GNorm = 3.4665, lr_0 = 6.3280e-04
Validation auc = 0.713146
Epoch 275
Validation auc = 0.716901
Epoch 276
Validation auc = 0.725352
Epoch 277
Validation auc = 0.729108
Epoch 278
Validation auc = 0.737559
Epoch 279
Loss = 2.7600e-01, PNorm = 44.9589, GNorm = 2.8883, lr_0 = 6.2739e-04
Validation auc = 0.739906
Epoch 280
Validation auc = 0.737559
Epoch 281
Validation auc = 0.735211
Epoch 282
Validation auc = 0.723944
Epoch 283
Validation auc = 0.722066
Epoch 284
Loss = 2.5160e-01, PNorm = 45.0346, GNorm = 1.8325, lr_0 = 6.2203e-04
Validation auc = 0.726761
Epoch 285
Validation auc = 0.728169
Epoch 286
Validation auc = 0.723944
Epoch 287
Validation auc = 0.723005
Epoch 288
Validation auc = 0.728169
Epoch 289
Loss = 2.3684e-01, PNorm = 45.1250, GNorm = 3.1984, lr_0 = 6.1671e-04
Validation auc = 0.733803
Epoch 290
Validation auc = 0.739906
Epoch 291
Validation auc = 0.745070
Epoch 292
Validation auc = 0.749765
Epoch 293
Validation auc = 0.751643
Epoch 294
Loss = 2.8611e-01, PNorm = 45.2016, GNorm = 2.7867, lr_0 = 6.1144e-04
Validation auc = 0.745540
Epoch 295
Validation auc = 0.743192
Epoch 296
Validation auc = 0.741315
Epoch 297
Validation auc = 0.743662
Epoch 298
Validation auc = 0.737559
Epoch 299
Loss = 2.0096e-01, PNorm = 45.2722, GNorm = 1.6293, lr_0 = 6.0621e-04
Validation auc = 0.729577
Model 0 best validation auc = 0.762911 on epoch 240
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.921734
Ensemble test auc = 0.921734
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.507981
Epoch 1
Validation auc = 0.681221
Epoch 2
Validation auc = 0.676056
Epoch 3
Validation auc = 0.688263
Epoch 4
Loss = 9.2551e-01, PNorm = 40.6666, GNorm = 9.8156, lr_0 = 6.5000e-04
Validation auc = 0.712676
Epoch 5
Validation auc = 0.705634
Epoch 6
Validation auc = 0.690141
Epoch 7
Validation auc = 0.677934
Epoch 8
Validation auc = 0.653521
Epoch 9
Loss = 8.5648e-01, PNorm = 40.7162, GNorm = 9.7049, lr_0 = 9.9743e-04
Validation auc = 0.631455
Epoch 10
Validation auc = 0.638028
Epoch 11
Validation auc = 0.617840
Epoch 12
Validation auc = 0.646009
Epoch 13
Validation auc = 0.629577
Epoch 14
Loss = 9.4990e-01, PNorm = 40.7607, GNorm = 7.5925, lr_0 = 9.8890e-04
Validation auc = 0.665258
Epoch 15
Validation auc = 0.697653
Epoch 16
Validation auc = 0.716901
Epoch 17
Validation auc = 0.678404
Epoch 18
Validation auc = 0.682160
Epoch 19
Loss = 7.0413e-01, PNorm = 40.7954, GNorm = 5.6336, lr_0 = 9.8045e-04
Validation auc = 0.703756
Epoch 20
Validation auc = 0.786854
Epoch 21
Validation auc = 0.835681
Epoch 22
Validation auc = 0.815023
Epoch 23
Validation auc = 0.711268
Epoch 24
Loss = 6.5052e-01, PNorm = 40.8276, GNorm = 5.5954, lr_0 = 9.7207e-04
Validation auc = 0.698122
Epoch 25
Validation auc = 0.700939
Epoch 26
Validation auc = 0.710798
Epoch 27
Validation auc = 0.694366
Epoch 28
Validation auc = 0.687793
Epoch 29
Loss = 5.5701e-01, PNorm = 40.8621, GNorm = 5.6477, lr_0 = 9.6376e-04
Validation auc = 0.676056
Epoch 30
Validation auc = 0.666197
Epoch 31
Validation auc = 0.667606
Epoch 32
Validation auc = 0.688732
Epoch 33
Validation auc = 0.731925
Epoch 34
Loss = 6.4863e-01, PNorm = 40.8961, GNorm = 5.7479, lr_0 = 9.5552e-04
Validation auc = 0.715493
Epoch 35
Validation auc = 0.692958
Epoch 36
Validation auc = 0.692488
Epoch 37
Validation auc = 0.688732
Epoch 38
Validation auc = 0.714554
Epoch 39
Loss = 5.7298e-01, PNorm = 40.9312, GNorm = 4.5262, lr_0 = 9.4735e-04
Validation auc = 0.747418
Epoch 40
Validation auc = 0.759155
Epoch 41
Validation auc = 0.740376
Epoch 42
Validation auc = 0.721127
Epoch 43
Validation auc = 0.726291
Epoch 44
Loss = 5.9993e-01, PNorm = 40.9665, GNorm = 5.0135, lr_0 = 9.3925e-04
Validation auc = 0.739437
Epoch 45
Validation auc = 0.732394
Epoch 46
Validation auc = 0.705634
Epoch 47
Validation auc = 0.694836
Epoch 48
Validation auc = 0.690141
Epoch 49
Loss = 5.9951e-01, PNorm = 41.0116, GNorm = 4.7488, lr_0 = 9.3122e-04
Validation auc = 0.686385
Epoch 50
Validation auc = 0.687324
Epoch 51
Validation auc = 0.692488
Epoch 52
Validation auc = 0.690610
Epoch 53
Validation auc = 0.684977
Epoch 54
Loss = 5.1889e-01, PNorm = 41.0583, GNorm = 3.1189, lr_0 = 9.2326e-04
Validation auc = 0.687324
Epoch 55
Validation auc = 0.689202
Epoch 56
Validation auc = 0.685915
Epoch 57
Validation auc = 0.681690
Epoch 58
Validation auc = 0.683568
Epoch 59
Loss = 6.2089e-01, PNorm = 41.1031, GNorm = 4.3007, lr_0 = 9.1537e-04
Validation auc = 0.656808
Epoch 60
Validation auc = 0.625822
Epoch 61
Validation auc = 0.605634
Epoch 62
Validation auc = 0.601408
Epoch 63
Validation auc = 0.603286
Epoch 64
Loss = 5.4972e-01, PNorm = 41.1586, GNorm = 4.7985, lr_0 = 9.0754e-04
Validation auc = 0.607042
Epoch 65
Validation auc = 0.615493
Epoch 66
Validation auc = 0.622535
Epoch 67
Validation auc = 0.623005
Epoch 68
Validation auc = 0.616432
Epoch 69
Loss = 5.5597e-01, PNorm = 41.2086, GNorm = 2.8667, lr_0 = 8.9978e-04
Validation auc = 0.607981
Epoch 70
Validation auc = 0.601408
Epoch 71
Validation auc = 0.599531
Epoch 72
Validation auc = 0.602817
Epoch 73
Validation auc = 0.598122
Epoch 74
Loss = 5.6000e-01, PNorm = 41.2612, GNorm = 3.0247, lr_0 = 8.9209e-04
Validation auc = 0.609390
Epoch 75
Validation auc = 0.622535
Epoch 76
Validation auc = 0.626761
Epoch 77
Validation auc = 0.623474
Epoch 78
Validation auc = 0.613146
Epoch 79
Loss = 4.7712e-01, PNorm = 41.3117, GNorm = 3.6181, lr_0 = 8.8447e-04
Validation auc = 0.623005
Epoch 80
Validation auc = 0.630047
Epoch 81
Validation auc = 0.631455
Epoch 82
Validation auc = 0.638498
Epoch 83
Validation auc = 0.645540
Epoch 84
Loss = 5.3522e-01, PNorm = 41.3727, GNorm = 2.7674, lr_0 = 8.7691e-04
Validation auc = 0.649765
Epoch 85
Validation auc = 0.642254
Epoch 86
Validation auc = 0.646009
Epoch 87
Validation auc = 0.657746
Epoch 88
Validation auc = 0.670423
Epoch 89
Loss = 5.7134e-01, PNorm = 41.4322, GNorm = 3.1952, lr_0 = 8.6941e-04
Validation auc = 0.676526
Epoch 90
Validation auc = 0.684038
Epoch 91
Validation auc = 0.690610
Epoch 92
Validation auc = 0.699061
Epoch 93
Validation auc = 0.698592
Epoch 94
Loss = 4.8302e-01, PNorm = 41.5126, GNorm = 3.6096, lr_0 = 8.6198e-04
Validation auc = 0.701408
Epoch 95
Validation auc = 0.696714
Epoch 96
Validation auc = 0.689671
Epoch 97
Validation auc = 0.686385
Epoch 98
Validation auc = 0.681690
Epoch 99
Loss = 4.8676e-01, PNorm = 41.5784, GNorm = 3.5674, lr_0 = 8.5461e-04
Validation auc = 0.688732
Epoch 100
Validation auc = 0.700000
Epoch 101
Validation auc = 0.707981
Epoch 102
Validation auc = 0.700939
Epoch 103
Validation auc = 0.700000
Epoch 104
Loss = 5.0546e-01, PNorm = 41.6618, GNorm = 3.1421, lr_0 = 8.4730e-04
Validation auc = 0.696714
Epoch 105
Validation auc = 0.693897
Epoch 106
Validation auc = 0.690141
Epoch 107
Validation auc = 0.673709
Epoch 108
Validation auc = 0.663850
Epoch 109
Loss = 4.4904e-01, PNorm = 41.7526, GNorm = 3.4140, lr_0 = 8.4006e-04
Validation auc = 0.666667
Epoch 110
Validation auc = 0.659155
Epoch 111
Validation auc = 0.646479
Epoch 112
Validation auc = 0.663850
Epoch 113
Validation auc = 0.661972
Epoch 114
Loss = 6.0960e-01, PNorm = 41.8488, GNorm = 4.5595, lr_0 = 8.3288e-04
Validation auc = 0.660094
Epoch 115
Validation auc = 0.659155
Epoch 116
Validation auc = 0.658216
Epoch 117
Validation auc = 0.683099
Epoch 118
Validation auc = 0.678873
Epoch 119
Loss = 4.0929e-01, PNorm = 41.9473, GNorm = 2.5357, lr_0 = 8.2576e-04
Validation auc = 0.666667
Epoch 120
Validation auc = 0.644601
Epoch 121
Validation auc = 0.642723
Epoch 122
Validation auc = 0.639437
Epoch 123
Validation auc = 0.646479
Epoch 124
Loss = 5.2056e-01, PNorm = 42.0355, GNorm = 3.8653, lr_0 = 8.1870e-04
Validation auc = 0.666197
Epoch 125
Validation auc = 0.681221
Epoch 126
Validation auc = 0.693897
Epoch 127
Validation auc = 0.696714
Epoch 128
Validation auc = 0.694836
Epoch 129
Loss = 4.1412e-01, PNorm = 42.1342, GNorm = 1.8463, lr_0 = 8.1170e-04
Validation auc = 0.692019
Epoch 130
Validation auc = 0.685915
Epoch 131
Validation auc = 0.703756
Epoch 132
Validation auc = 0.708920
Epoch 133
Validation auc = 0.719718
Epoch 134
Loss = 4.9161e-01, PNorm = 42.2235, GNorm = 3.0690, lr_0 = 8.0476e-04
Validation auc = 0.724413
Epoch 135
Validation auc = 0.727700
Epoch 136
Validation auc = 0.729108
Epoch 137
Validation auc = 0.722535
Epoch 138
Validation auc = 0.710329
Epoch 139
Loss = 4.2085e-01, PNorm = 42.3326, GNorm = 4.0519, lr_0 = 7.9788e-04
Validation auc = 0.712207
Epoch 140
Validation auc = 0.708451
Epoch 141
Validation auc = 0.703286
Epoch 142
Validation auc = 0.721596
Epoch 143
Validation auc = 0.714554
Epoch 144
Loss = 4.4986e-01, PNorm = 42.4493, GNorm = 2.5090, lr_0 = 7.9106e-04
Validation auc = 0.726291
Epoch 145
Validation auc = 0.722535
Epoch 146
Validation auc = 0.721596
Epoch 147
Validation auc = 0.726761
Epoch 148
Validation auc = 0.735681
Epoch 149
Loss = 4.9168e-01, PNorm = 42.5552, GNorm = 3.1605, lr_0 = 7.8430e-04
Validation auc = 0.733803
Epoch 150
Validation auc = 0.739906
Epoch 151
Validation auc = 0.741784
Epoch 152
Validation auc = 0.735211
Epoch 153
Validation auc = 0.730516
Epoch 154
Loss = 3.2871e-01, PNorm = 42.6718, GNorm = 3.0211, lr_0 = 7.7759e-04
Validation auc = 0.724413
Epoch 155
Validation auc = 0.725352
Epoch 156
Validation auc = 0.732394
Epoch 157
Validation auc = 0.731455
Epoch 158
Validation auc = 0.750704
Epoch 159
Loss = 3.9675e-01, PNorm = 42.7862, GNorm = 4.1472, lr_0 = 7.7095e-04
Validation auc = 0.751643
Epoch 160
Validation auc = 0.734742
Epoch 161
Validation auc = 0.731925
Epoch 162
Validation auc = 0.730516
Epoch 163
Validation auc = 0.727230
Epoch 164
Loss = 4.0452e-01, PNorm = 42.8710, GNorm = 4.3885, lr_0 = 7.6436e-04
Validation auc = 0.727700
Epoch 165
Validation auc = 0.730047
Epoch 166
Validation auc = 0.739437
Epoch 167
Validation auc = 0.747887
Epoch 168
Validation auc = 0.748357
Epoch 169
Loss = 3.0051e-01, PNorm = 42.9763, GNorm = 2.3212, lr_0 = 7.5782e-04
Validation auc = 0.740376
Epoch 170
Validation auc = 0.731455
Epoch 171
Validation auc = 0.723944
Epoch 172
Validation auc = 0.713146
Epoch 173
Validation auc = 0.707512
Epoch 174
Loss = 3.6762e-01, PNorm = 43.0772, GNorm = 3.2309, lr_0 = 7.5134e-04
Validation auc = 0.711268
Epoch 175
Validation auc = 0.719249
Epoch 176
Validation auc = 0.721127
Epoch 177
Validation auc = 0.733333
Epoch 178
Validation auc = 0.739437
Epoch 179
Loss = 3.5989e-01, PNorm = 43.1597, GNorm = 2.8168, lr_0 = 7.4492e-04
Validation auc = 0.739906
Epoch 180
Validation auc = 0.743662
Epoch 181
Validation auc = 0.745070
Epoch 182
Validation auc = 0.746948
Epoch 183
Validation auc = 0.733803
Epoch 184
Loss = 4.0502e-01, PNorm = 43.2428, GNorm = 2.9526, lr_0 = 7.3855e-04
Validation auc = 0.723944
Epoch 185
Validation auc = 0.721127
Epoch 186
Validation auc = 0.718779
Epoch 187
Validation auc = 0.722066
Epoch 188
Validation auc = 0.733333
Epoch 189
Loss = 2.6426e-01, PNorm = 43.3387, GNorm = 1.7376, lr_0 = 7.3224e-04
Validation auc = 0.746009
Epoch 190
Validation auc = 0.753521
Epoch 191
Validation auc = 0.756808
Epoch 192
Validation auc = 0.765258
Epoch 193
Validation auc = 0.761033
Epoch 194
Loss = 2.9796e-01, PNorm = 43.4258, GNorm = 2.2773, lr_0 = 7.2598e-04
Validation auc = 0.762441
Epoch 195
Validation auc = 0.758216
Epoch 196
Validation auc = 0.750704
Epoch 197
Validation auc = 0.745070
Epoch 198
Validation auc = 0.740376
Epoch 199
Loss = 3.3403e-01, PNorm = 43.5011, GNorm = 1.9868, lr_0 = 7.1977e-04
Validation auc = 0.744131
Epoch 200
Validation auc = 0.746948
Epoch 201
Validation auc = 0.750704
Epoch 202
Validation auc = 0.759155
Epoch 203
Validation auc = 0.765258
Epoch 204
Loss = 2.7899e-01, PNorm = 43.5770, GNorm = 2.0935, lr_0 = 7.1362e-04
Validation auc = 0.769484
Epoch 205
Validation auc = 0.761502
Epoch 206
Validation auc = 0.749765
Epoch 207
Validation auc = 0.749296
Epoch 208
Validation auc = 0.748826
Epoch 209
Loss = 3.3478e-01, PNorm = 43.6682, GNorm = 2.7318, lr_0 = 7.0752e-04
Validation auc = 0.747418
Epoch 210
Validation auc = 0.747887
Epoch 211
Validation auc = 0.749296
Epoch 212
Validation auc = 0.753521
Epoch 213
Validation auc = 0.757277
Epoch 214
Loss = 3.6373e-01, PNorm = 43.7502, GNorm = 3.2662, lr_0 = 7.0147e-04
Validation auc = 0.754930
Epoch 215
Validation auc = 0.748826
Epoch 216
Validation auc = 0.739437
Epoch 217
Validation auc = 0.734272
Epoch 218
Validation auc = 0.733333
Epoch 219
Loss = 2.8889e-01, PNorm = 43.8212, GNorm = 1.4596, lr_0 = 6.9548e-04
Validation auc = 0.729577
Epoch 220
Validation auc = 0.721596
Epoch 221
Validation auc = 0.717840
Epoch 222
Validation auc = 0.715962
Epoch 223
Validation auc = 0.714554
Epoch 224
Loss = 2.7272e-01, PNorm = 43.8994, GNorm = 3.6426, lr_0 = 6.8953e-04
Validation auc = 0.718310
Epoch 225
Validation auc = 0.723005
Epoch 226
Validation auc = 0.727700
Epoch 227
Validation auc = 0.737559
Epoch 228
Validation auc = 0.740376
Epoch 229
Loss = 3.6931e-01, PNorm = 43.9625, GNorm = 2.3944, lr_0 = 6.8364e-04
Validation auc = 0.741315
Epoch 230
Validation auc = 0.744131
Epoch 231
Validation auc = 0.753521
Epoch 232
Validation auc = 0.766667
Epoch 233
Validation auc = 0.768545
Epoch 234
Loss = 4.3187e-01, PNorm = 44.0352, GNorm = 2.0812, lr_0 = 6.7779e-04
Validation auc = 0.762441
Epoch 235
Validation auc = 0.765258
Epoch 236
Validation auc = 0.766197
Epoch 237
Validation auc = 0.767136
Epoch 238
Validation auc = 0.759155
Epoch 239
Loss = 3.2616e-01, PNorm = 44.1099, GNorm = 3.8610, lr_0 = 6.7200e-04
Validation auc = 0.758685
Epoch 240
Validation auc = 0.753521
Epoch 241
Validation auc = 0.759624
Epoch 242
Validation auc = 0.758685
Epoch 243
Validation auc = 0.751174
Epoch 244
Loss = 2.8388e-01, PNorm = 44.1916, GNorm = 1.6452, lr_0 = 6.6625e-04
Validation auc = 0.747887
Epoch 245
Validation auc = 0.735211
Epoch 246
Validation auc = 0.732864
Epoch 247
Validation auc = 0.732864
Epoch 248
Validation auc = 0.737089
Epoch 249
Loss = 3.0000e-01, PNorm = 44.2691, GNorm = 2.0629, lr_0 = 6.6056e-04
Validation auc = 0.737559
Epoch 250
Validation auc = 0.740376
Epoch 251
Validation auc = 0.745070
Epoch 252
Validation auc = 0.740845
Epoch 253
Validation auc = 0.740376
Epoch 254
Loss = 3.1042e-01, PNorm = 44.3411, GNorm = 3.2057, lr_0 = 6.5491e-04
Validation auc = 0.740845
Epoch 255
Validation auc = 0.742254
Epoch 256
Validation auc = 0.739437
Epoch 257
Validation auc = 0.736620
Epoch 258
Validation auc = 0.736620
Epoch 259
Loss = 2.8796e-01, PNorm = 44.4145, GNorm = 2.3813, lr_0 = 6.4931e-04
Validation auc = 0.732864
Epoch 260
Validation auc = 0.738498
Epoch 261
Validation auc = 0.748826
Epoch 262
Validation auc = 0.745540
Epoch 263
Validation auc = 0.738028
Epoch 264
Loss = 2.8911e-01, PNorm = 44.4792, GNorm = 2.8896, lr_0 = 6.4376e-04
Validation auc = 0.734742
Epoch 265
Validation auc = 0.731925
Epoch 266
Validation auc = 0.735211
Epoch 267
Validation auc = 0.735211
Epoch 268
Validation auc = 0.735681
Epoch 269
Loss = 2.7430e-01, PNorm = 44.5443, GNorm = 1.3776, lr_0 = 6.3826e-04
Validation auc = 0.737559
Epoch 270
Validation auc = 0.739906
Epoch 271
Validation auc = 0.740845
Epoch 272
Validation auc = 0.743192
Epoch 273
Validation auc = 0.744601
Epoch 274
Loss = 2.1225e-01, PNorm = 44.6094, GNorm = 1.6614, lr_0 = 6.3280e-04
Validation auc = 0.739437
Epoch 275
Validation auc = 0.738028
Epoch 276
Validation auc = 0.741315
Epoch 277
Validation auc = 0.744601
Epoch 278
Validation auc = 0.745540
Epoch 279
Loss = 2.6938e-01, PNorm = 44.6797, GNorm = 1.4283, lr_0 = 6.2739e-04
Validation auc = 0.737559
Epoch 280
Validation auc = 0.730516
Epoch 281
Validation auc = 0.730516
Epoch 282
Validation auc = 0.737089
Epoch 283
Validation auc = 0.753521
Epoch 284
Loss = 3.3574e-01, PNorm = 44.7613, GNorm = 3.8172, lr_0 = 6.2203e-04
Validation auc = 0.755869
Epoch 285
Validation auc = 0.750235
Epoch 286
Validation auc = 0.746948
Epoch 287
Validation auc = 0.748357
Epoch 288
Validation auc = 0.751174
Epoch 289
Loss = 2.5106e-01, PNorm = 44.8312, GNorm = 1.5080, lr_0 = 6.1671e-04
Validation auc = 0.758216
Epoch 290
Validation auc = 0.756338
Epoch 291
Validation auc = 0.758216
Epoch 292
Validation auc = 0.755399
Epoch 293
Validation auc = 0.751174
Epoch 294
Loss = 2.6446e-01, PNorm = 44.9100, GNorm = 2.0535, lr_0 = 6.1144e-04
Validation auc = 0.753052
Epoch 295
Validation auc = 0.746948
Epoch 296
Validation auc = 0.746479
Epoch 297
Validation auc = 0.744131
Epoch 298
Validation auc = 0.736620
Epoch 299
Loss = 2.6127e-01, PNorm = 44.9815, GNorm = 4.4577, lr_0 = 6.0621e-04
Validation auc = 0.735681
Model 0 best validation auc = 0.835681 on epoch 21
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.797110
Ensemble test auc = 0.797110
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.511268
Epoch 1
Validation auc = 0.688263
Epoch 2
Validation auc = 0.684038
Epoch 3
Validation auc = 0.703286
Epoch 4
Loss = 1.0335e+00, PNorm = 40.6655, GNorm = 13.8268, lr_0 = 6.5000e-04
Validation auc = 0.715962
Epoch 5
Validation auc = 0.721596
Epoch 6
Validation auc = 0.666667
Epoch 7
Validation auc = 0.661972
Epoch 8
Validation auc = 0.644601
Epoch 9
Loss = 1.0849e+00, PNorm = 40.7127, GNorm = 12.4416, lr_0 = 9.9743e-04
Validation auc = 0.644601
Epoch 10
Validation auc = 0.657746
Epoch 11
Validation auc = 0.652582
Epoch 12
Validation auc = 0.650704
Epoch 13
Validation auc = 0.638967
Epoch 14
Loss = 7.3438e-01, PNorm = 40.7623, GNorm = 6.9317, lr_0 = 9.8890e-04
Validation auc = 0.639437
Epoch 15
Validation auc = 0.650704
Epoch 16
Validation auc = 0.652113
Epoch 17
Validation auc = 0.650704
Epoch 18
Validation auc = 0.688263
Epoch 19
Loss = 7.3504e-01, PNorm = 40.8110, GNorm = 5.9748, lr_0 = 9.8045e-04
Validation auc = 0.710798
Epoch 20
Validation auc = 0.697183
Epoch 21
Validation auc = 0.694366
Epoch 22
Validation auc = 0.687324
Epoch 23
Validation auc = 0.689202
Epoch 24
Loss = 7.7267e-01, PNorm = 40.8503, GNorm = 8.7644, lr_0 = 9.7207e-04
Validation auc = 0.704225
Epoch 25
Validation auc = 0.701878
Epoch 26
Validation auc = 0.707042
Epoch 27
Validation auc = 0.713615
Epoch 28
Validation auc = 0.696714
Epoch 29
Loss = 6.7391e-01, PNorm = 40.8792, GNorm = 6.4853, lr_0 = 9.6376e-04
Validation auc = 0.722535
Epoch 30
Validation auc = 0.664319
Epoch 31
Validation auc = 0.660563
Epoch 32
Validation auc = 0.658685
Epoch 33
Validation auc = 0.647887
Epoch 34
Loss = 5.8075e-01, PNorm = 40.9269, GNorm = 3.7780, lr_0 = 9.5552e-04
Validation auc = 0.654460
Epoch 35
Validation auc = 0.639437
Epoch 36
Validation auc = 0.624413
Epoch 37
Validation auc = 0.617371
Epoch 38
Validation auc = 0.606573
Epoch 39
Loss = 6.7670e-01, PNorm = 40.9526, GNorm = 5.6055, lr_0 = 9.4735e-04
Validation auc = 0.609390
Epoch 40
Validation auc = 0.616901
Epoch 41
Validation auc = 0.643192
Epoch 42
Validation auc = 0.664789
Epoch 43
Validation auc = 0.661972
Epoch 44
Loss = 6.2557e-01, PNorm = 40.9860, GNorm = 3.8241, lr_0 = 9.3925e-04
Validation auc = 0.649765
Epoch 45
Validation auc = 0.641315
Epoch 46
Validation auc = 0.636150
Epoch 47
Validation auc = 0.642723
Epoch 48
Validation auc = 0.644131
Epoch 49
Loss = 5.5161e-01, PNorm = 41.0275, GNorm = 3.8299, lr_0 = 9.3122e-04
Validation auc = 0.644601
Epoch 50
Validation auc = 0.640376
Epoch 51
Validation auc = 0.643192
Epoch 52
Validation auc = 0.649296
Epoch 53
Validation auc = 0.645540
Epoch 54
Loss = 6.3878e-01, PNorm = 41.0698, GNorm = 5.2715, lr_0 = 9.2326e-04
Validation auc = 0.638028
Epoch 55
Validation auc = 0.634272
Epoch 56
Validation auc = 0.642723
Epoch 57
Validation auc = 0.644131
Epoch 58
Validation auc = 0.650235
Epoch 59
Loss = 6.4173e-01, PNorm = 41.1079, GNorm = 3.5508, lr_0 = 9.1537e-04
Validation auc = 0.684507
Epoch 60
Validation auc = 0.685915
Epoch 61
Validation auc = 0.673239
Epoch 62
Validation auc = 0.672770
Epoch 63
Validation auc = 0.682160
Epoch 64
Loss = 5.6578e-01, PNorm = 41.1633, GNorm = 4.0758, lr_0 = 9.0754e-04
Validation auc = 0.687793
Epoch 65
Validation auc = 0.680282
Epoch 66
Validation auc = 0.663850
Epoch 67
Validation auc = 0.649765
Epoch 68
Validation auc = 0.642723
Epoch 69
Loss = 5.6712e-01, PNorm = 41.2104, GNorm = 3.5649, lr_0 = 8.9978e-04
Validation auc = 0.642254
Epoch 70
Validation auc = 0.647418
Epoch 71
Validation auc = 0.648357
Epoch 72
Validation auc = 0.644601
Epoch 73
Validation auc = 0.651174
Epoch 74
Loss = 5.9027e-01, PNorm = 41.2907, GNorm = 3.5333, lr_0 = 8.9209e-04
Validation auc = 0.653052
Epoch 75
Validation auc = 0.652582
Epoch 76
Validation auc = 0.661502
Epoch 77
Validation auc = 0.674178
Epoch 78
Validation auc = 0.690141
Epoch 79
Loss = 5.9295e-01, PNorm = 41.3316, GNorm = 4.2430, lr_0 = 8.8447e-04
Validation auc = 0.691549
Epoch 80
Validation auc = 0.694836
Epoch 81
Validation auc = 0.668545
Epoch 82
Validation auc = 0.638498
Epoch 83
Validation auc = 0.643662
Epoch 84
Loss = 5.5442e-01, PNorm = 41.4069, GNorm = 5.3977, lr_0 = 8.7691e-04
Validation auc = 0.655869
Epoch 85
Validation auc = 0.662441
Epoch 86
Validation auc = 0.666667
Epoch 87
Validation auc = 0.666197
Epoch 88
Validation auc = 0.661033
Epoch 89
Loss = 4.4261e-01, PNorm = 41.4816, GNorm = 3.2324, lr_0 = 8.6941e-04
Validation auc = 0.651643
Epoch 90
Validation auc = 0.640845
Epoch 91
Validation auc = 0.631925
Epoch 92
Validation auc = 0.615962
Epoch 93
Validation auc = 0.615962
Epoch 94
Loss = 4.8953e-01, PNorm = 41.5556, GNorm = 3.5319, lr_0 = 8.6198e-04
Validation auc = 0.620188
Epoch 95
Validation auc = 0.621596
Epoch 96
Validation auc = 0.619718
Epoch 97
Validation auc = 0.608451
Epoch 98
Validation auc = 0.606573
Epoch 99
Loss = 5.8039e-01, PNorm = 41.6272, GNorm = 3.8166, lr_0 = 8.5461e-04
Validation auc = 0.619249
Epoch 100
Validation auc = 0.626761
Epoch 101
Validation auc = 0.645070
Epoch 102
Validation auc = 0.664319
Epoch 103
Validation auc = 0.664789
Epoch 104
Loss = 4.5526e-01, PNorm = 41.7101, GNorm = 2.8701, lr_0 = 8.4730e-04
Validation auc = 0.666197
Epoch 105
Validation auc = 0.659155
Epoch 106
Validation auc = 0.644131
Epoch 107
Validation auc = 0.635211
Epoch 108
Validation auc = 0.638498
Epoch 109
Loss = 4.9369e-01, PNorm = 41.7676, GNorm = 2.2677, lr_0 = 8.4006e-04
Validation auc = 0.644131
Epoch 110
Validation auc = 0.650704
Epoch 111
Validation auc = 0.649765
Epoch 112
Validation auc = 0.638498
Epoch 113
Validation auc = 0.643192
Epoch 114
Loss = 4.8779e-01, PNorm = 41.8602, GNorm = 3.7279, lr_0 = 8.3288e-04
Validation auc = 0.652113
Epoch 115
Validation auc = 0.669014
Epoch 116
Validation auc = 0.663380
Epoch 117
Validation auc = 0.668545
Epoch 118
Validation auc = 0.668075
Epoch 119
Loss = 5.8264e-01, PNorm = 41.9553, GNorm = 2.3273, lr_0 = 8.2576e-04
Validation auc = 0.661033
Epoch 120
Validation auc = 0.660563
Epoch 121
Validation auc = 0.671362
Epoch 122
Validation auc = 0.676056
Epoch 123
Validation auc = 0.680282
Epoch 124
Loss = 4.0334e-01, PNorm = 42.0540, GNorm = 3.0817, lr_0 = 8.1870e-04
Validation auc = 0.671831
Epoch 125
Validation auc = 0.668545
Epoch 126
Validation auc = 0.666667
Epoch 127
Validation auc = 0.673239
Epoch 128
Validation auc = 0.670892
Epoch 129
Loss = 4.8197e-01, PNorm = 42.1586, GNorm = 2.7118, lr_0 = 8.1170e-04
Validation auc = 0.683099
Epoch 130
Validation auc = 0.700939
Epoch 131
Validation auc = 0.701408
Epoch 132
Validation auc = 0.700000
Epoch 133
Validation auc = 0.701878
Epoch 134
Loss = 3.7057e-01, PNorm = 42.2632, GNorm = 3.0221, lr_0 = 8.0476e-04
Validation auc = 0.706573
Epoch 135
Validation auc = 0.708920
Epoch 136
Validation auc = 0.703286
Epoch 137
Validation auc = 0.708920
Epoch 138
Validation auc = 0.713615
Epoch 139
Loss = 4.2169e-01, PNorm = 42.3812, GNorm = 2.1885, lr_0 = 7.9788e-04
Validation auc = 0.709390
Epoch 140
Validation auc = 0.707042
Epoch 141
Validation auc = 0.700000
Epoch 142
Validation auc = 0.701408
Epoch 143
Validation auc = 0.704225
Epoch 144
Loss = 3.9695e-01, PNorm = 42.4665, GNorm = 2.3187, lr_0 = 7.9106e-04
Validation auc = 0.709859
Epoch 145
Validation auc = 0.714554
Epoch 146
Validation auc = 0.710798
Epoch 147
Validation auc = 0.716901
Epoch 148
Validation auc = 0.714554
Epoch 149
Loss = 4.3793e-01, PNorm = 42.5498, GNorm = 3.9545, lr_0 = 7.8430e-04
Validation auc = 0.723474
Epoch 150
Validation auc = 0.728169
Epoch 151
Validation auc = 0.734742
Epoch 152
Validation auc = 0.738967
Epoch 153
Validation auc = 0.738498
Epoch 154
Loss = 4.4117e-01, PNorm = 42.6555, GNorm = 2.8707, lr_0 = 7.7759e-04
Validation auc = 0.727700
Epoch 155
Validation auc = 0.724883
Epoch 156
Validation auc = 0.716432
Epoch 157
Validation auc = 0.714554
Epoch 158
Validation auc = 0.703286
Epoch 159
Loss = 4.5156e-01, PNorm = 42.7668, GNorm = 3.4066, lr_0 = 7.7095e-04
Validation auc = 0.707042
Epoch 160
Validation auc = 0.721596
Epoch 161
Validation auc = 0.730047
Epoch 162
Validation auc = 0.734742
Epoch 163
Validation auc = 0.735681
Epoch 164
Loss = 4.3522e-01, PNorm = 42.8561, GNorm = 3.6832, lr_0 = 7.6436e-04
Validation auc = 0.714085
Epoch 165
Validation auc = 0.702817
Epoch 166
Validation auc = 0.703286
Epoch 167
Validation auc = 0.693897
Epoch 168
Validation auc = 0.693427
Epoch 169
Loss = 3.3454e-01, PNorm = 42.9635, GNorm = 2.6490, lr_0 = 7.5782e-04
Validation auc = 0.694836
Epoch 170
Validation auc = 0.697183
Epoch 171
Validation auc = 0.704695
Epoch 172
Validation auc = 0.713146
Epoch 173
Validation auc = 0.713146
Epoch 174
Loss = 3.3167e-01, PNorm = 43.0684, GNorm = 1.9595, lr_0 = 7.5134e-04
Validation auc = 0.719718
Epoch 175
Validation auc = 0.718310
Epoch 176
Validation auc = 0.714085
Epoch 177
Validation auc = 0.713146
Epoch 178
Validation auc = 0.712207
Epoch 179
Loss = 3.0406e-01, PNorm = 43.1787, GNorm = 2.6592, lr_0 = 7.4492e-04
Validation auc = 0.728638
Epoch 180
Validation auc = 0.746948
Epoch 181
Validation auc = 0.751643
Epoch 182
Validation auc = 0.740376
Epoch 183
Validation auc = 0.738967
Epoch 184
Loss = 4.1302e-01, PNorm = 43.2710, GNorm = 2.9524, lr_0 = 7.3855e-04
Validation auc = 0.746479
Epoch 185
Validation auc = 0.748826
Epoch 186
Validation auc = 0.740376
Epoch 187
Validation auc = 0.725822
Epoch 188
Validation auc = 0.705634
Epoch 189
Loss = 4.0835e-01, PNorm = 43.3506, GNorm = 2.3178, lr_0 = 7.3224e-04
Validation auc = 0.715493
Epoch 190
Validation auc = 0.719718
Epoch 191
Validation auc = 0.719718
Epoch 192
Validation auc = 0.724883
Epoch 193
Validation auc = 0.724883
Epoch 194
Loss = 3.8194e-01, PNorm = 43.4408, GNorm = 1.8036, lr_0 = 7.2598e-04
Validation auc = 0.723944
Epoch 195
Validation auc = 0.718310
Epoch 196
Validation auc = 0.726761
Epoch 197
Validation auc = 0.726291
Epoch 198
Validation auc = 0.729577
Epoch 199
Loss = 3.3246e-01, PNorm = 43.5347, GNorm = 2.2120, lr_0 = 7.1977e-04
Validation auc = 0.719718
Epoch 200
Validation auc = 0.710798
Epoch 201
Validation auc = 0.699531
Epoch 202
Validation auc = 0.708451
Epoch 203
Validation auc = 0.709859
Epoch 204
Loss = 3.4608e-01, PNorm = 43.6281, GNorm = 3.8036, lr_0 = 7.1362e-04
Validation auc = 0.719249
Epoch 205
Validation auc = 0.724883
Epoch 206
Validation auc = 0.726291
Epoch 207
Validation auc = 0.726291
Epoch 208
Validation auc = 0.727230
Epoch 209
Loss = 3.7296e-01, PNorm = 43.7045, GNorm = 2.8805, lr_0 = 7.0752e-04
Validation auc = 0.727700
Epoch 210
Validation auc = 0.725352
Epoch 211
Validation auc = 0.735211
Epoch 212
Validation auc = 0.730986
Epoch 213
Validation auc = 0.724413
Epoch 214
Loss = 3.0332e-01, PNorm = 43.8134, GNorm = 1.9838, lr_0 = 7.0147e-04
Validation auc = 0.725352
Epoch 215
Validation auc = 0.721596
Epoch 216
Validation auc = 0.724883
Epoch 217
Validation auc = 0.720657
Epoch 218
Validation auc = 0.724413
Epoch 219
Loss = 3.6710e-01, PNorm = 43.9164, GNorm = 3.2902, lr_0 = 6.9548e-04
Validation auc = 0.720657
Epoch 220
Validation auc = 0.725822
Epoch 221
Validation auc = 0.725822
Epoch 222
Validation auc = 0.717371
Epoch 223
Validation auc = 0.719249
Epoch 224
Loss = 3.0545e-01, PNorm = 43.9850, GNorm = 2.4454, lr_0 = 6.8953e-04
Validation auc = 0.722535
Epoch 225
Validation auc = 0.718779
Epoch 226
Validation auc = 0.720657
Epoch 227
Validation auc = 0.721127
Epoch 228
Validation auc = 0.722066
Epoch 229
Loss = 2.8623e-01, PNorm = 44.0787, GNorm = 2.3642, lr_0 = 6.8364e-04
Validation auc = 0.725352
Epoch 230
Validation auc = 0.726291
Epoch 231
Validation auc = 0.723944
Epoch 232
Validation auc = 0.727230
Epoch 233
Validation auc = 0.727230
Epoch 234
Loss = 3.4498e-01, PNorm = 44.1853, GNorm = 4.1308, lr_0 = 6.7779e-04
Validation auc = 0.724883
Epoch 235
Validation auc = 0.720188
Epoch 236
Validation auc = 0.717371
Epoch 237
Validation auc = 0.715023
Epoch 238
Validation auc = 0.715493
Epoch 239
Loss = 2.1002e-01, PNorm = 44.2865, GNorm = 2.1728, lr_0 = 6.7200e-04
Validation auc = 0.717840
Epoch 240
Validation auc = 0.725352
Epoch 241
Validation auc = 0.729108
Epoch 242
Validation auc = 0.732864
Epoch 243
Validation auc = 0.736150
Epoch 244
Loss = 2.1420e-01, PNorm = 44.4062, GNorm = 2.2416, lr_0 = 6.6625e-04
Validation auc = 0.736150
Epoch 245
Validation auc = 0.737089
Epoch 246
Validation auc = 0.737089
Epoch 247
Validation auc = 0.743662
Epoch 248
Validation auc = 0.743662
Epoch 249
Loss = 2.1093e-01, PNorm = 44.4928, GNorm = 1.9247, lr_0 = 6.6056e-04
Validation auc = 0.741315
Epoch 250
Validation auc = 0.742254
Epoch 251
Validation auc = 0.745070
Epoch 252
Validation auc = 0.739437
Epoch 253
Validation auc = 0.741784
Epoch 254
Loss = 2.4774e-01, PNorm = 44.5642, GNorm = 4.3796, lr_0 = 6.5491e-04
Validation auc = 0.740845
Epoch 255
Validation auc = 0.737559
Epoch 256
Validation auc = 0.732394
Epoch 257
Validation auc = 0.729577
Epoch 258
Validation auc = 0.727230
Epoch 259
Loss = 2.6047e-01, PNorm = 44.6649, GNorm = 2.1701, lr_0 = 6.4931e-04
Validation auc = 0.730516
Epoch 260
Validation auc = 0.736620
Epoch 261
Validation auc = 0.732864
Epoch 262
Validation auc = 0.729577
Epoch 263
Validation auc = 0.727700
Epoch 264
Loss = 1.8792e-01, PNorm = 44.7490, GNorm = 2.5597, lr_0 = 6.4376e-04
Validation auc = 0.728638
Epoch 265
Validation auc = 0.723944
Epoch 266
Validation auc = 0.718310
Epoch 267
Validation auc = 0.711268
Epoch 268
Validation auc = 0.709390
Epoch 269
Loss = 3.5221e-01, PNorm = 44.8222, GNorm = 2.6000, lr_0 = 6.3826e-04
Validation auc = 0.708451
Epoch 270
Validation auc = 0.709859
Epoch 271
Validation auc = 0.710798
Epoch 272
Validation auc = 0.708920
Epoch 273
Validation auc = 0.704225
Epoch 274
Loss = 3.1966e-01, PNorm = 44.9041, GNorm = 4.6139, lr_0 = 6.3280e-04
Validation auc = 0.707512
Epoch 275
Validation auc = 0.706103
Epoch 276
Validation auc = 0.708451
Epoch 277
Validation auc = 0.710798
Epoch 278
Validation auc = 0.716432
Epoch 279
Loss = 2.2695e-01, PNorm = 44.9770, GNorm = 1.6474, lr_0 = 6.2739e-04
Validation auc = 0.720188
Epoch 280
Validation auc = 0.722066
Epoch 281
Validation auc = 0.727230
Epoch 282
Validation auc = 0.731925
Epoch 283
Validation auc = 0.739437
Epoch 284
Loss = 2.6159e-01, PNorm = 45.0653, GNorm = 3.8780, lr_0 = 6.2203e-04
Validation auc = 0.744131
Epoch 285
Validation auc = 0.748357
Epoch 286
Validation auc = 0.747887
Epoch 287
Validation auc = 0.743662
Epoch 288
Validation auc = 0.739437
Epoch 289
Loss = 2.6213e-01, PNorm = 45.1488, GNorm = 2.3032, lr_0 = 6.1671e-04
Validation auc = 0.741315
Epoch 290
Validation auc = 0.737559
Epoch 291
Validation auc = 0.738028
Epoch 292
Validation auc = 0.736620
Epoch 293
Validation auc = 0.733803
Epoch 294
Loss = 1.8566e-01, PNorm = 45.2323, GNorm = 3.1061, lr_0 = 6.1144e-04
Validation auc = 0.732394
Epoch 295
Validation auc = 0.726291
Epoch 296
Validation auc = 0.720188
Epoch 297
Validation auc = 0.723944
Epoch 298
Validation auc = 0.727700
Epoch 299
Loss = 1.8340e-01, PNorm = 45.3164, GNorm = 2.9613, lr_0 = 6.0621e-04
Validation auc = 0.732394
Model 0 best validation auc = 0.751643 on epoch 181
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.918724
Ensemble test auc = 0.918724
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.603286
Epoch 1
Validation auc = 0.682629
Epoch 2
Validation auc = 0.666667
Epoch 3
Validation auc = 0.664789
Epoch 4
Loss = 1.1578e+00, PNorm = 40.6675, GNorm = 18.6631, lr_0 = 6.5000e-04
Validation auc = 0.663850
Epoch 5
Validation auc = 0.662911
Epoch 6
Validation auc = 0.654930
Epoch 7
Validation auc = 0.643192
Epoch 8
Validation auc = 0.621127
Epoch 9
Loss = 8.0703e-01, PNorm = 40.7168, GNorm = 6.9227, lr_0 = 9.9743e-04
Validation auc = 0.627700
Epoch 10
Validation auc = 0.630986
Epoch 11
Validation auc = 0.641315
Epoch 12
Validation auc = 0.638967
Epoch 13
Validation auc = 0.645070
Epoch 14
Loss = 6.0281e-01, PNorm = 40.7580, GNorm = 7.3325, lr_0 = 9.8890e-04
Validation auc = 0.664789
Epoch 15
Validation auc = 0.675117
Epoch 16
Validation auc = 0.676526
Epoch 17
Validation auc = 0.675587
Epoch 18
Validation auc = 0.676526
Epoch 19
Loss = 6.3585e-01, PNorm = 40.8023, GNorm = 6.2515, lr_0 = 9.8045e-04
Validation auc = 0.665258
Epoch 20
Validation auc = 0.649296
Epoch 21
Validation auc = 0.661033
Epoch 22
Validation auc = 0.667606
Epoch 23
Validation auc = 0.688263
Epoch 24
Loss = 7.1434e-01, PNorm = 40.8367, GNorm = 5.4259, lr_0 = 9.7207e-04
Validation auc = 0.714085
Epoch 25
Validation auc = 0.705634
Epoch 26
Validation auc = 0.686854
Epoch 27
Validation auc = 0.686854
Epoch 28
Validation auc = 0.686385
Epoch 29
Loss = 6.4502e-01, PNorm = 40.8571, GNorm = 5.1571, lr_0 = 9.6376e-04
Validation auc = 0.682160
Epoch 30
Validation auc = 0.680282
Epoch 31
Validation auc = 0.690610
Epoch 32
Validation auc = 0.682160
Epoch 33
Validation auc = 0.674648
Epoch 34
Loss = 6.6455e-01, PNorm = 40.8930, GNorm = 5.7153, lr_0 = 9.5552e-04
Validation auc = 0.677934
Epoch 35
Validation auc = 0.678404
Epoch 36
Validation auc = 0.669953
Epoch 37
Validation auc = 0.661502
Epoch 38
Validation auc = 0.668075
Epoch 39
Loss = 6.6550e-01, PNorm = 40.9279, GNorm = 4.4305, lr_0 = 9.4735e-04
Validation auc = 0.684977
Epoch 40
Validation auc = 0.689671
Epoch 41
Validation auc = 0.688732
Epoch 42
Validation auc = 0.680282
Epoch 43
Validation auc = 0.679343
Epoch 44
Loss = 5.7585e-01, PNorm = 40.9649, GNorm = 4.5449, lr_0 = 9.3925e-04
Validation auc = 0.685915
Epoch 45
Validation auc = 0.683568
Epoch 46
Validation auc = 0.692958
Epoch 47
Validation auc = 0.697653
Epoch 48
Validation auc = 0.692488
Epoch 49
Loss = 5.6212e-01, PNorm = 41.0062, GNorm = 3.7860, lr_0 = 9.3122e-04
Validation auc = 0.689202
Epoch 50
Validation auc = 0.680751
Epoch 51
Validation auc = 0.679343
Epoch 52
Validation auc = 0.682160
Epoch 53
Validation auc = 0.686854
Epoch 54
Loss = 6.0842e-01, PNorm = 41.0402, GNorm = 4.9117, lr_0 = 9.2326e-04
Validation auc = 0.683568
Epoch 55
Validation auc = 0.670892
Epoch 56
Validation auc = 0.656808
Epoch 57
Validation auc = 0.656338
Epoch 58
Validation auc = 0.648826
Epoch 59
Loss = 5.3344e-01, PNorm = 41.0928, GNorm = 2.5048, lr_0 = 9.1537e-04
Validation auc = 0.653521
Epoch 60
Validation auc = 0.644601
Epoch 61
Validation auc = 0.646009
Epoch 62
Validation auc = 0.648826
Epoch 63
Validation auc = 0.648826
Epoch 64
Loss = 5.5027e-01, PNorm = 41.1510, GNorm = 2.9258, lr_0 = 9.0754e-04
Validation auc = 0.646479
Epoch 65
Validation auc = 0.638967
Epoch 66
Validation auc = 0.647418
Epoch 67
Validation auc = 0.647887
Epoch 68
Validation auc = 0.647887
Epoch 69
Loss = 5.2007e-01, PNorm = 41.2037, GNorm = 2.7923, lr_0 = 8.9978e-04
Validation auc = 0.641784
Epoch 70
Validation auc = 0.634272
Epoch 71
Validation auc = 0.629577
Epoch 72
Validation auc = 0.631925
Epoch 73
Validation auc = 0.637089
Epoch 74
Loss = 5.7075e-01, PNorm = 41.2577, GNorm = 3.4166, lr_0 = 8.9209e-04
Validation auc = 0.640845
Epoch 75
Validation auc = 0.640376
Epoch 76
Validation auc = 0.639437
Epoch 77
Validation auc = 0.640376
Epoch 78
Validation auc = 0.642254
Epoch 79
Loss = 5.1405e-01, PNorm = 41.3365, GNorm = 3.2384, lr_0 = 8.8447e-04
Validation auc = 0.640845
Epoch 80
Validation auc = 0.633803
Epoch 81
Validation auc = 0.636150
Epoch 82
Validation auc = 0.639437
Epoch 83
Validation auc = 0.647887
Epoch 84
Loss = 4.8568e-01, PNorm = 41.3885, GNorm = 2.7124, lr_0 = 8.7691e-04
Validation auc = 0.653521
Epoch 85
Validation auc = 0.655399
Epoch 86
Validation auc = 0.655399
Epoch 87
Validation auc = 0.655399
Epoch 88
Validation auc = 0.658216
Epoch 89
Loss = 4.8943e-01, PNorm = 41.4778, GNorm = 2.8749, lr_0 = 8.6941e-04
Validation auc = 0.671831
Epoch 90
Validation auc = 0.673709
Epoch 91
Validation auc = 0.652113
Epoch 92
Validation auc = 0.648826
Epoch 93
Validation auc = 0.640376
Epoch 94
Loss = 5.6406e-01, PNorm = 41.5380, GNorm = 2.6476, lr_0 = 8.6198e-04
Validation auc = 0.642254
Epoch 95
Validation auc = 0.649765
Epoch 96
Validation auc = 0.660563
Epoch 97
Validation auc = 0.656808
Epoch 98
Validation auc = 0.656338
Epoch 99
Loss = 5.2838e-01, PNorm = 41.6108, GNorm = 3.2421, lr_0 = 8.5461e-04
Validation auc = 0.672770
Epoch 100
Validation auc = 0.680751
Epoch 101
Validation auc = 0.678873
Epoch 102
Validation auc = 0.682160
Epoch 103
Validation auc = 0.683568
Epoch 104
Loss = 5.5900e-01, PNorm = 41.6980, GNorm = 3.9225, lr_0 = 8.4730e-04
Validation auc = 0.685915
Epoch 105
Validation auc = 0.686385
Epoch 106
Validation auc = 0.690141
Epoch 107
Validation auc = 0.708451
Epoch 108
Validation auc = 0.730516
Epoch 109
Loss = 4.6490e-01, PNorm = 41.7674, GNorm = 3.6123, lr_0 = 8.4006e-04
Validation auc = 0.726291
Epoch 110
Validation auc = 0.697183
Epoch 111
Validation auc = 0.676995
Epoch 112
Validation auc = 0.665728
Epoch 113
Validation auc = 0.670892
Epoch 114
Loss = 5.2028e-01, PNorm = 41.8708, GNorm = 3.4822, lr_0 = 8.3288e-04
Validation auc = 0.687793
Epoch 115
Validation auc = 0.706573
Epoch 116
Validation auc = 0.697183
Epoch 117
Validation auc = 0.689671
Epoch 118
Validation auc = 0.688732
Epoch 119
Loss = 4.5549e-01, PNorm = 41.9655, GNorm = 2.7803, lr_0 = 8.2576e-04
Validation auc = 0.684038
Epoch 120
Validation auc = 0.670892
Epoch 121
Validation auc = 0.671831
Epoch 122
Validation auc = 0.669014
Epoch 123
Validation auc = 0.674178
Epoch 124
Loss = 4.3298e-01, PNorm = 42.0772, GNorm = 2.1209, lr_0 = 8.1870e-04
Validation auc = 0.679343
Epoch 125
Validation auc = 0.677465
Epoch 126
Validation auc = 0.681221
Epoch 127
Validation auc = 0.690610
Epoch 128
Validation auc = 0.687793
Epoch 129
Loss = 4.8987e-01, PNorm = 42.1762, GNorm = 4.0184, lr_0 = 8.1170e-04
Validation auc = 0.683099
Epoch 130
Validation auc = 0.683568
Epoch 131
Validation auc = 0.682629
Epoch 132
Validation auc = 0.690610
Epoch 133
Validation auc = 0.696714
Epoch 134
Loss = 3.9693e-01, PNorm = 42.2874, GNorm = 3.0544, lr_0 = 8.0476e-04
Validation auc = 0.698122
Epoch 135
Validation auc = 0.699531
Epoch 136
Validation auc = 0.699531
Epoch 137
Validation auc = 0.702347
Epoch 138
Validation auc = 0.707981
Epoch 139
Loss = 3.9907e-01, PNorm = 42.4010, GNorm = 2.3495, lr_0 = 7.9788e-04
Validation auc = 0.710329
Epoch 140
Validation auc = 0.719249
Epoch 141
Validation auc = 0.727230
Epoch 142
Validation auc = 0.718310
Epoch 143
Validation auc = 0.711268
Epoch 144
Loss = 4.4270e-01, PNorm = 42.5147, GNorm = 2.9390, lr_0 = 7.9106e-04
Validation auc = 0.701878
Epoch 145
Validation auc = 0.697653
Epoch 146
Validation auc = 0.698592
Epoch 147
Validation auc = 0.707512
Epoch 148
Validation auc = 0.711737
Epoch 149
Loss = 5.1841e-01, PNorm = 42.6306, GNorm = 3.3875, lr_0 = 7.8430e-04
Validation auc = 0.715962
Epoch 150
Validation auc = 0.716901
Epoch 151
Validation auc = 0.715962
Epoch 152
Validation auc = 0.716901
Epoch 153
Validation auc = 0.723474
Epoch 154
Loss = 3.8656e-01, PNorm = 42.7469, GNorm = 3.3456, lr_0 = 7.7759e-04
Validation auc = 0.723944
Epoch 155
Validation auc = 0.722066
Epoch 156
Validation auc = 0.725822
Epoch 157
Validation auc = 0.722535
Epoch 158
Validation auc = 0.724413
Epoch 159
Loss = 3.2216e-01, PNorm = 42.8652, GNorm = 2.8300, lr_0 = 7.7095e-04
Validation auc = 0.723005
Epoch 160
Validation auc = 0.719718
Epoch 161
Validation auc = 0.721596
Epoch 162
Validation auc = 0.719718
Epoch 163
Validation auc = 0.715493
Epoch 164
Loss = 3.6263e-01, PNorm = 42.9752, GNorm = 2.0726, lr_0 = 7.6436e-04
Validation auc = 0.717840
Epoch 165
Validation auc = 0.711737
Epoch 166
Validation auc = 0.710329
Epoch 167
Validation auc = 0.706103
Epoch 168
Validation auc = 0.710798
Epoch 169
Loss = 3.3312e-01, PNorm = 43.0876, GNorm = 1.4397, lr_0 = 7.5782e-04
Validation auc = 0.712207
Epoch 170
Validation auc = 0.710798
Epoch 171
Validation auc = 0.705164
Epoch 172
Validation auc = 0.702347
Epoch 173
Validation auc = 0.702817
Epoch 174
Loss = 3.9319e-01, PNorm = 43.1969, GNorm = 2.2667, lr_0 = 7.5134e-04
Validation auc = 0.707042
Epoch 175
Validation auc = 0.705634
Epoch 176
Validation auc = 0.709390
Epoch 177
Validation auc = 0.711268
Epoch 178
Validation auc = 0.712676
Epoch 179
Loss = 3.5268e-01, PNorm = 43.3053, GNorm = 2.1145, lr_0 = 7.4492e-04
Validation auc = 0.719718
Epoch 180
Validation auc = 0.720188
Epoch 181
Validation auc = 0.722535
Epoch 182
Validation auc = 0.719249
Epoch 183
Validation auc = 0.722535
Epoch 184
Loss = 4.0613e-01, PNorm = 43.4103, GNorm = 3.4378, lr_0 = 7.3855e-04
Validation auc = 0.726291
Epoch 185
Validation auc = 0.722535
Epoch 186
Validation auc = 0.717371
Epoch 187
Validation auc = 0.709859
Epoch 188
Validation auc = 0.707981
Epoch 189
Loss = 3.5514e-01, PNorm = 43.5181, GNorm = 2.6670, lr_0 = 7.3224e-04
Validation auc = 0.714085
Epoch 190
Validation auc = 0.719718
Epoch 191
Validation auc = 0.722535
Epoch 192
Validation auc = 0.735681
Epoch 193
Validation auc = 0.734272
Epoch 194
Loss = 3.9230e-01, PNorm = 43.6279, GNorm = 2.4198, lr_0 = 7.2598e-04
Validation auc = 0.724883
Epoch 195
Validation auc = 0.717371
Epoch 196
Validation auc = 0.715023
Epoch 197
Validation auc = 0.708451
Epoch 198
Validation auc = 0.707042
Epoch 199
Loss = 3.3719e-01, PNorm = 43.7318, GNorm = 1.8758, lr_0 = 7.1977e-04
Validation auc = 0.710329
Epoch 200
Validation auc = 0.706573
Epoch 201
Validation auc = 0.705164
Epoch 202
Validation auc = 0.708451
Epoch 203
Validation auc = 0.718310
Epoch 204
Loss = 3.5699e-01, PNorm = 43.8292, GNorm = 3.3207, lr_0 = 7.1362e-04
Validation auc = 0.726761
Epoch 205
Validation auc = 0.726761
Epoch 206
Validation auc = 0.726291
Epoch 207
Validation auc = 0.726761
Epoch 208
Validation auc = 0.724413
Epoch 209
Loss = 2.8859e-01, PNorm = 43.9209, GNorm = 1.9776, lr_0 = 7.0752e-04
Validation auc = 0.723474
Epoch 210
Validation auc = 0.718779
Epoch 211
Validation auc = 0.719718
Epoch 212
Validation auc = 0.725822
Epoch 213
Validation auc = 0.730986
Epoch 214
Loss = 3.6578e-01, PNorm = 44.0188, GNorm = 3.5555, lr_0 = 7.0147e-04
Validation auc = 0.726291
Epoch 215
Validation auc = 0.718779
Epoch 216
Validation auc = 0.721596
Epoch 217
Validation auc = 0.720657
Epoch 218
Validation auc = 0.723944
Epoch 219
Loss = 3.2868e-01, PNorm = 44.1161, GNorm = 2.0936, lr_0 = 6.9548e-04
Validation auc = 0.723474
Epoch 220
Validation auc = 0.724883
Epoch 221
Validation auc = 0.725822
Epoch 222
Validation auc = 0.728638
Epoch 223
Validation auc = 0.731455
Epoch 224
Loss = 4.2067e-01, PNorm = 44.2279, GNorm = 3.6196, lr_0 = 6.8953e-04
Validation auc = 0.725352
Epoch 225
Validation auc = 0.723944
Epoch 226
Validation auc = 0.725822
Epoch 227
Validation auc = 0.727700
Epoch 228
Validation auc = 0.730047
Epoch 229
Loss = 3.5825e-01, PNorm = 44.3240, GNorm = 3.9799, lr_0 = 6.8364e-04
Validation auc = 0.735211
Epoch 230
Validation auc = 0.732864
Epoch 231
Validation auc = 0.732864
Epoch 232
Validation auc = 0.733803
Epoch 233
Validation auc = 0.739437
Epoch 234
Loss = 3.2741e-01, PNorm = 44.4289, GNorm = 1.8655, lr_0 = 6.7779e-04
Validation auc = 0.738967
Epoch 235
Validation auc = 0.739437
Epoch 236
Validation auc = 0.737559
Epoch 237
Validation auc = 0.734742
Epoch 238
Validation auc = 0.737089
Epoch 239
Loss = 3.4168e-01, PNorm = 44.5322, GNorm = 2.8227, lr_0 = 6.7200e-04
Validation auc = 0.734742
Epoch 240
Validation auc = 0.738028
Epoch 241
Validation auc = 0.738498
Epoch 242
Validation auc = 0.737089
Epoch 243
Validation auc = 0.733803
Epoch 244
Loss = 2.7933e-01, PNorm = 44.6158, GNorm = 2.6784, lr_0 = 6.6625e-04
Validation auc = 0.731455
Epoch 245
Validation auc = 0.732394
Epoch 246
Validation auc = 0.728638
Epoch 247
Validation auc = 0.728638
Epoch 248
Validation auc = 0.727230
Epoch 249
Loss = 3.2775e-01, PNorm = 44.6892, GNorm = 7.4501, lr_0 = 6.6056e-04
Validation auc = 0.729108
Epoch 250
Validation auc = 0.729108
Epoch 251
Validation auc = 0.730986
Epoch 252
Validation auc = 0.729577
Epoch 253
Validation auc = 0.730986
Epoch 254
Loss = 2.8893e-01, PNorm = 44.7835, GNorm = 3.2858, lr_0 = 6.5491e-04
Validation auc = 0.731925
Epoch 255
Validation auc = 0.732864
Epoch 256
Validation auc = 0.735681
Epoch 257
Validation auc = 0.735211
Epoch 258
Validation auc = 0.736620
Epoch 259
Loss = 3.0361e-01, PNorm = 44.8716, GNorm = 2.5285, lr_0 = 6.4931e-04
Validation auc = 0.739906
Epoch 260
Validation auc = 0.741784
Epoch 261
Validation auc = 0.738028
Epoch 262
Validation auc = 0.738498
Epoch 263
Validation auc = 0.739906
Epoch 264
Loss = 3.1136e-01, PNorm = 44.9499, GNorm = 2.4712, lr_0 = 6.4376e-04
Validation auc = 0.746948
Epoch 265
Validation auc = 0.749296
Epoch 266
Validation auc = 0.744601
Epoch 267
Validation auc = 0.744131
Epoch 268
Validation auc = 0.736620
Epoch 269
Loss = 2.5056e-01, PNorm = 45.0418, GNorm = 3.1355, lr_0 = 6.3826e-04
Validation auc = 0.735211
Epoch 270
Validation auc = 0.723944
Epoch 271
Validation auc = 0.719718
Epoch 272
Validation auc = 0.721127
Epoch 273
Validation auc = 0.722066
Epoch 274
Loss = 2.6467e-01, PNorm = 45.1052, GNorm = 2.6963, lr_0 = 6.3280e-04
Validation auc = 0.726291
Epoch 275
Validation auc = 0.727230
Epoch 276
Validation auc = 0.729577
Epoch 277
Validation auc = 0.725352
Epoch 278
Validation auc = 0.727700
Epoch 279
Loss = 2.8762e-01, PNorm = 45.1775, GNorm = 1.6946, lr_0 = 6.2739e-04
Validation auc = 0.726291
Epoch 280
Validation auc = 0.729577
Epoch 281
Validation auc = 0.731925
Epoch 282
Validation auc = 0.730986
Epoch 283
Validation auc = 0.732394
Epoch 284
Loss = 2.6083e-01, PNorm = 45.2759, GNorm = 2.3677, lr_0 = 6.2203e-04
Validation auc = 0.732864
Epoch 285
Validation auc = 0.739437
Epoch 286
Validation auc = 0.738967
Epoch 287
Validation auc = 0.735681
Epoch 288
Validation auc = 0.723005
Epoch 289
Loss = 1.8965e-01, PNorm = 45.3599, GNorm = 2.6017, lr_0 = 6.1671e-04
Validation auc = 0.715962
Epoch 290
Validation auc = 0.714554
Epoch 291
Validation auc = 0.721596
Epoch 292
Validation auc = 0.726291
Epoch 293
Validation auc = 0.730047
Epoch 294
Loss = 1.6455e-01, PNorm = 45.4562, GNorm = 1.4983, lr_0 = 6.1144e-04
Validation auc = 0.732394
Epoch 295
Validation auc = 0.742254
Epoch 296
Validation auc = 0.745540
Epoch 297
Validation auc = 0.741784
Epoch 298
Validation auc = 0.735681
Epoch 299
Loss = 1.6366e-01, PNorm = 45.5460, GNorm = 1.6271, lr_0 = 6.0621e-04
Validation auc = 0.734742
Model 0 best validation auc = 0.749296 on epoch 265
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.919326
Ensemble test auc = 0.919326
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.536150
Epoch 1
Validation auc = 0.675117
Epoch 2
Validation auc = 0.665728
Epoch 3
Validation auc = 0.670892
Epoch 4
Loss = 1.1533e+00, PNorm = 40.6663, GNorm = 16.0173, lr_0 = 6.5000e-04
Validation auc = 0.692488
Epoch 5
Validation auc = 0.704695
Epoch 6
Validation auc = 0.703756
Epoch 7
Validation auc = 0.704695
Epoch 8
Validation auc = 0.712676
Epoch 9
Loss = 8.8233e-01, PNorm = 40.7134, GNorm = 9.5232, lr_0 = 9.9743e-04
Validation auc = 0.735681
Epoch 10
Validation auc = 0.696244
Epoch 11
Validation auc = 0.669953
Epoch 12
Validation auc = 0.666197
Epoch 13
Validation auc = 0.657746
Epoch 14
Loss = 8.9626e-01, PNorm = 40.7581, GNorm = 7.0457, lr_0 = 9.8890e-04
Validation auc = 0.694836
Epoch 15
Validation auc = 0.670423
Epoch 16
Validation auc = 0.659155
Epoch 17
Validation auc = 0.658685
Epoch 18
Validation auc = 0.651643
Epoch 19
Loss = 7.1981e-01, PNorm = 40.8034, GNorm = 6.7761, lr_0 = 9.8045e-04
Validation auc = 0.634272
Epoch 20
Validation auc = 0.637089
Epoch 21
Validation auc = 0.633333
Epoch 22
Validation auc = 0.642723
Epoch 23
Validation auc = 0.650704
Epoch 24
Loss = 6.1751e-01, PNorm = 40.8444, GNorm = 7.1741, lr_0 = 9.7207e-04
Validation auc = 0.648357
Epoch 25
Validation auc = 0.661972
Epoch 26
Validation auc = 0.672300
Epoch 27
Validation auc = 0.684507
Epoch 28
Validation auc = 0.700469
Epoch 29
Loss = 5.6580e-01, PNorm = 40.8728, GNorm = 4.9620, lr_0 = 9.6376e-04
Validation auc = 0.667136
Epoch 30
Validation auc = 0.663850
Epoch 31
Validation auc = 0.666197
Epoch 32
Validation auc = 0.674648
Epoch 33
Validation auc = 0.680751
Epoch 34
Loss = 7.1216e-01, PNorm = 40.9066, GNorm = 6.3080, lr_0 = 9.5552e-04
Validation auc = 0.692958
Epoch 35
Validation auc = 0.689202
Epoch 36
Validation auc = 0.685915
Epoch 37
Validation auc = 0.680282
Epoch 38
Validation auc = 0.677465
Epoch 39
Loss = 6.6779e-01, PNorm = 40.9509, GNorm = 4.4193, lr_0 = 9.4735e-04
Validation auc = 0.665728
Epoch 40
Validation auc = 0.642723
Epoch 41
Validation auc = 0.618779
Epoch 42
Validation auc = 0.630516
Epoch 43
Validation auc = 0.644131
Epoch 44
Loss = 5.5403e-01, PNorm = 40.9773, GNorm = 5.3221, lr_0 = 9.3925e-04
Validation auc = 0.642254
Epoch 45
Validation auc = 0.637559
Epoch 46
Validation auc = 0.624413
Epoch 47
Validation auc = 0.667606
Epoch 48
Validation auc = 0.682629
Epoch 49
Loss = 5.5389e-01, PNorm = 41.0108, GNorm = 3.2024, lr_0 = 9.3122e-04
Validation auc = 0.669014
Epoch 50
Validation auc = 0.653521
Epoch 51
Validation auc = 0.644131
Epoch 52
Validation auc = 0.646009
Epoch 53
Validation auc = 0.654460
Epoch 54
Loss = 5.4156e-01, PNorm = 41.0342, GNorm = 3.5037, lr_0 = 9.2326e-04
Validation auc = 0.653521
Epoch 55
Validation auc = 0.655869
Epoch 56
Validation auc = 0.650235
Epoch 57
Validation auc = 0.655869
Epoch 58
Validation auc = 0.652113
Epoch 59
Loss = 5.5974e-01, PNorm = 41.0695, GNorm = 4.2071, lr_0 = 9.1537e-04
Validation auc = 0.646009
Epoch 60
Validation auc = 0.644601
Epoch 61
Validation auc = 0.639437
Epoch 62
Validation auc = 0.637089
Epoch 63
Validation auc = 0.649765
Epoch 64
Loss = 5.7852e-01, PNorm = 41.1212, GNorm = 3.7879, lr_0 = 9.0754e-04
Validation auc = 0.659155
Epoch 65
Validation auc = 0.659624
Epoch 66
Validation auc = 0.657746
Epoch 67
Validation auc = 0.657746
Epoch 68
Validation auc = 0.655399
Epoch 69
Loss = 5.3268e-01, PNorm = 41.1519, GNorm = 3.4951, lr_0 = 8.9978e-04
Validation auc = 0.644601
Epoch 70
Validation auc = 0.642254
Epoch 71
Validation auc = 0.644131
Epoch 72
Validation auc = 0.649296
Epoch 73
Validation auc = 0.650235
Epoch 74
Loss = 6.3504e-01, PNorm = 41.2065, GNorm = 2.7114, lr_0 = 8.9209e-04
Validation auc = 0.635211
Epoch 75
Validation auc = 0.622535
Epoch 76
Validation auc = 0.628638
Epoch 77
Validation auc = 0.639437
Epoch 78
Validation auc = 0.660563
Epoch 79
Loss = 5.4777e-01, PNorm = 41.2713, GNorm = 3.4111, lr_0 = 8.8447e-04
Validation auc = 0.665258
Epoch 80
Validation auc = 0.670423
Epoch 81
Validation auc = 0.666667
Epoch 82
Validation auc = 0.655869
Epoch 83
Validation auc = 0.649765
Epoch 84
Loss = 5.1738e-01, PNorm = 41.3113, GNorm = 2.7891, lr_0 = 8.7691e-04
Validation auc = 0.638028
Epoch 85
Validation auc = 0.635211
Epoch 86
Validation auc = 0.636620
Epoch 87
Validation auc = 0.639437
Epoch 88
Validation auc = 0.643662
Epoch 89
Loss = 4.7493e-01, PNorm = 41.3699, GNorm = 2.6341, lr_0 = 8.6941e-04
Validation auc = 0.647887
Epoch 90
Validation auc = 0.648826
Epoch 91
Validation auc = 0.666197
Epoch 92
Validation auc = 0.674648
Epoch 93
Validation auc = 0.677465
Epoch 94
Loss = 5.5329e-01, PNorm = 41.4377, GNorm = 3.8990, lr_0 = 8.6198e-04
Validation auc = 0.681221
Epoch 95
Validation auc = 0.692958
Epoch 96
Validation auc = 0.697183
Epoch 97
Validation auc = 0.686854
Epoch 98
Validation auc = 0.667136
Epoch 99
Loss = 5.0480e-01, PNorm = 41.5209, GNorm = 2.3855, lr_0 = 8.5461e-04
Validation auc = 0.642723
Epoch 100
Validation auc = 0.646948
Epoch 101
Validation auc = 0.646009
Epoch 102
Validation auc = 0.656338
Epoch 103
Validation auc = 0.666667
Epoch 104
Loss = 4.5554e-01, PNorm = 41.5956, GNorm = 1.9447, lr_0 = 8.4730e-04
Validation auc = 0.669014
Epoch 105
Validation auc = 0.661502
Epoch 106
Validation auc = 0.657277
Epoch 107
Validation auc = 0.661972
Epoch 108
Validation auc = 0.671362
Epoch 109
Loss = 4.9870e-01, PNorm = 41.6633, GNorm = 3.1835, lr_0 = 8.4006e-04
Validation auc = 0.678404
Epoch 110
Validation auc = 0.680282
Epoch 111
Validation auc = 0.680751
Epoch 112
Validation auc = 0.681221
Epoch 113
Validation auc = 0.672300
Epoch 114
Loss = 5.2354e-01, PNorm = 41.7399, GNorm = 2.3512, lr_0 = 8.3288e-04
Validation auc = 0.667606
Epoch 115
Validation auc = 0.662911
Epoch 116
Validation auc = 0.654460
Epoch 117
Validation auc = 0.657277
Epoch 118
Validation auc = 0.672300
Epoch 119
Loss = 5.0476e-01, PNorm = 41.8358, GNorm = 3.2639, lr_0 = 8.2576e-04
Validation auc = 0.693897
Epoch 120
Validation auc = 0.697183
Epoch 121
Validation auc = 0.700939
Epoch 122
Validation auc = 0.680282
Epoch 123
Validation auc = 0.678873
Epoch 124
Loss = 4.0270e-01, PNorm = 41.9268, GNorm = 2.1858, lr_0 = 8.1870e-04
Validation auc = 0.679812
Epoch 125
Validation auc = 0.682160
Epoch 126
Validation auc = 0.687793
Epoch 127
Validation auc = 0.697653
Epoch 128
Validation auc = 0.692488
Epoch 129
Loss = 4.5132e-01, PNorm = 42.0280, GNorm = 3.1844, lr_0 = 8.1170e-04
Validation auc = 0.691080
Epoch 130
Validation auc = 0.687793
Epoch 131
Validation auc = 0.692488
Epoch 132
Validation auc = 0.707042
Epoch 133
Validation auc = 0.698592
Epoch 134
Loss = 4.5658e-01, PNorm = 42.1328, GNorm = 3.6524, lr_0 = 8.0476e-04
Validation auc = 0.701878
Epoch 135
Validation auc = 0.692958
Epoch 136
Validation auc = 0.704695
Epoch 137
Validation auc = 0.706103
Epoch 138
Validation auc = 0.707042
Epoch 139
Loss = 4.8805e-01, PNorm = 42.2145, GNorm = 3.3067, lr_0 = 7.9788e-04
Validation auc = 0.709859
Epoch 140
Validation auc = 0.715962
Epoch 141
Validation auc = 0.733333
Epoch 142
Validation auc = 0.737559
Epoch 143
Validation auc = 0.734742
Epoch 144
Loss = 4.4108e-01, PNorm = 42.3014, GNorm = 3.2396, lr_0 = 7.9106e-04
Validation auc = 0.730047
Epoch 145
Validation auc = 0.731455
Epoch 146
Validation auc = 0.723005
Epoch 147
Validation auc = 0.730986
Epoch 148
Validation auc = 0.737559
Epoch 149
Loss = 4.3145e-01, PNorm = 42.4123, GNorm = 3.2729, lr_0 = 7.8430e-04
Validation auc = 0.745070
Epoch 150
Validation auc = 0.763850
Epoch 151
Validation auc = 0.756338
Epoch 152
Validation auc = 0.752113
Epoch 153
Validation auc = 0.743192
Epoch 154
Loss = 4.0937e-01, PNorm = 42.4988, GNorm = 4.3696, lr_0 = 7.7759e-04
Validation auc = 0.743192
Epoch 155
Validation auc = 0.743192
Epoch 156
Validation auc = 0.722535
Epoch 157
Validation auc = 0.707512
Epoch 158
Validation auc = 0.696714
Epoch 159
Loss = 4.1342e-01, PNorm = 42.5807, GNorm = 3.4529, lr_0 = 7.7095e-04
Validation auc = 0.691549
Epoch 160
Validation auc = 0.700469
Epoch 161
Validation auc = 0.710798
Epoch 162
Validation auc = 0.738028
Epoch 163
Validation auc = 0.736150
Epoch 164
Loss = 3.7821e-01, PNorm = 42.6576, GNorm = 2.0713, lr_0 = 7.6436e-04
Validation auc = 0.742723
Epoch 165
Validation auc = 0.738967
Epoch 166
Validation auc = 0.733333
Epoch 167
Validation auc = 0.724413
Epoch 168
Validation auc = 0.722066
Epoch 169
Loss = 3.7799e-01, PNorm = 42.7566, GNorm = 2.0476, lr_0 = 7.5782e-04
Validation auc = 0.722066
Epoch 170
Validation auc = 0.720188
Epoch 171
Validation auc = 0.728638
Epoch 172
Validation auc = 0.729108
Epoch 173
Validation auc = 0.735681
Epoch 174
Loss = 3.8103e-01, PNorm = 42.8446, GNorm = 2.5770, lr_0 = 7.5134e-04
Validation auc = 0.727700
Epoch 175
Validation auc = 0.733333
Epoch 176
Validation auc = 0.730516
Epoch 177
Validation auc = 0.729108
Epoch 178
Validation auc = 0.734742
Epoch 179
Loss = 4.0257e-01, PNorm = 42.9218, GNorm = 2.5655, lr_0 = 7.4492e-04
Validation auc = 0.750235
Epoch 180
Validation auc = 0.741315
Epoch 181
Validation auc = 0.725352
Epoch 182
Validation auc = 0.710798
Epoch 183
Validation auc = 0.714554
Epoch 184
Loss = 3.5298e-01, PNorm = 42.9909, GNorm = 1.9254, lr_0 = 7.3855e-04
Validation auc = 0.726761
Epoch 185
Validation auc = 0.730516
Epoch 186
Validation auc = 0.729577
Epoch 187
Validation auc = 0.715493
Epoch 188
Validation auc = 0.702817
Epoch 189
Loss = 3.3940e-01, PNorm = 43.0687, GNorm = 2.4999, lr_0 = 7.3224e-04
Validation auc = 0.690610
Epoch 190
Validation auc = 0.695305
Epoch 191
Validation auc = 0.707512
Epoch 192
Validation auc = 0.710329
Epoch 193
Validation auc = 0.717371
Epoch 194
Loss = 3.5784e-01, PNorm = 43.1454, GNorm = 4.3917, lr_0 = 7.2598e-04
Validation auc = 0.717371
Epoch 195
Validation auc = 0.718310
Epoch 196
Validation auc = 0.719718
Epoch 197
Validation auc = 0.733333
Epoch 198
Validation auc = 0.730986
Epoch 199
Loss = 3.5423e-01, PNorm = 43.2422, GNorm = 2.2942, lr_0 = 7.1977e-04
Validation auc = 0.715962
Epoch 200
Validation auc = 0.706103
Epoch 201
Validation auc = 0.705164
Epoch 202
Validation auc = 0.716432
Epoch 203
Validation auc = 0.723474
Epoch 204
Loss = 3.6390e-01, PNorm = 43.3271, GNorm = 2.4569, lr_0 = 7.1362e-04
Validation auc = 0.711737
Epoch 205
Validation auc = 0.711268
Epoch 206
Validation auc = 0.715023
Epoch 207
Validation auc = 0.719249
Epoch 208
Validation auc = 0.723474
Epoch 209
Loss = 2.7273e-01, PNorm = 43.4115, GNorm = 3.6572, lr_0 = 7.0752e-04
Validation auc = 0.723474
Epoch 210
Validation auc = 0.719249
Epoch 211
Validation auc = 0.723005
Epoch 212
Validation auc = 0.734272
Epoch 213
Validation auc = 0.731455
Epoch 214
Loss = 3.7233e-01, PNorm = 43.4889, GNorm = 2.5126, lr_0 = 7.0147e-04
Validation auc = 0.721596
Epoch 215
Validation auc = 0.714085
Epoch 216
Validation auc = 0.711737
Epoch 217
Validation auc = 0.716901
Epoch 218
Validation auc = 0.724883
Epoch 219
Loss = 3.0311e-01, PNorm = 43.5604, GNorm = 3.0114, lr_0 = 6.9548e-04
Validation auc = 0.730516
Epoch 220
Validation auc = 0.729577
Epoch 221
Validation auc = 0.733333
Epoch 222
Validation auc = 0.735681
Epoch 223
Validation auc = 0.735681
Epoch 224
Loss = 3.3344e-01, PNorm = 43.6485, GNorm = 2.1899, lr_0 = 6.8953e-04
Validation auc = 0.737559
Epoch 225
Validation auc = 0.743192
Epoch 226
Validation auc = 0.749296
Epoch 227
Validation auc = 0.750235
Epoch 228
Validation auc = 0.754460
Epoch 229
Loss = 3.1298e-01, PNorm = 43.7215, GNorm = 2.8020, lr_0 = 6.8364e-04
Validation auc = 0.744601
Epoch 230
Validation auc = 0.742723
Epoch 231
Validation auc = 0.741315
Epoch 232
Validation auc = 0.740845
Epoch 233
Validation auc = 0.743192
Epoch 234
Loss = 2.9451e-01, PNorm = 43.8068, GNorm = 2.1586, lr_0 = 6.7779e-04
Validation auc = 0.748826
Epoch 235
Validation auc = 0.752582
Epoch 236
Validation auc = 0.754930
Epoch 237
Validation auc = 0.757277
Epoch 238
Validation auc = 0.756338
Epoch 239
Loss = 3.2507e-01, PNorm = 43.8931, GNorm = 2.4427, lr_0 = 6.7200e-04
Validation auc = 0.752582
Epoch 240
Validation auc = 0.741784
Epoch 241
Validation auc = 0.730047
Epoch 242
Validation auc = 0.742723
Epoch 243
Validation auc = 0.737559
Epoch 244
Loss = 2.8611e-01, PNorm = 43.9835, GNorm = 2.9700, lr_0 = 6.6625e-04
Validation auc = 0.728638
Epoch 245
Validation auc = 0.722535
Epoch 246
Validation auc = 0.722535
Epoch 247
Validation auc = 0.729108
Epoch 248
Validation auc = 0.733333
Epoch 249
Loss = 2.2796e-01, PNorm = 44.0774, GNorm = 3.1706, lr_0 = 6.6056e-04
Validation auc = 0.724883
Epoch 250
Validation auc = 0.720188
Epoch 251
Validation auc = 0.730986
Epoch 252
Validation auc = 0.731455
Epoch 253
Validation auc = 0.720188
Epoch 254
Loss = 2.7918e-01, PNorm = 44.1538, GNorm = 2.0900, lr_0 = 6.5491e-04
Validation auc = 0.711268
Epoch 255
Validation auc = 0.721596
Epoch 256
Validation auc = 0.727230
Epoch 257
Validation auc = 0.715023
Epoch 258
Validation auc = 0.705164
Epoch 259
Loss = 3.4441e-01, PNorm = 44.2343, GNorm = 2.7034, lr_0 = 6.4931e-04
Validation auc = 0.710329
Epoch 260
Validation auc = 0.716432
Epoch 261
Validation auc = 0.719718
Epoch 262
Validation auc = 0.726291
Epoch 263
Validation auc = 0.728169
Epoch 264
Loss = 2.9932e-01, PNorm = 44.3186, GNorm = 2.1079, lr_0 = 6.4376e-04
Validation auc = 0.723944
Epoch 265
Validation auc = 0.717371
Epoch 266
Validation auc = 0.722066
Epoch 267
Validation auc = 0.724413
Epoch 268
Validation auc = 0.730047
Epoch 269
Loss = 2.8537e-01, PNorm = 44.3823, GNorm = 2.3619, lr_0 = 6.3826e-04
Validation auc = 0.730516
Epoch 270
Validation auc = 0.735211
Epoch 271
Validation auc = 0.735211
Epoch 272
Validation auc = 0.738498
Epoch 273
Validation auc = 0.738967
Epoch 274
Loss = 2.1924e-01, PNorm = 44.4664, GNorm = 1.8456, lr_0 = 6.3280e-04
Validation auc = 0.737089
Epoch 275
Validation auc = 0.735211
Epoch 276
Validation auc = 0.735681
Epoch 277
Validation auc = 0.740376
Epoch 278
Validation auc = 0.746479
Epoch 279
Loss = 2.8099e-01, PNorm = 44.5584, GNorm = 4.6434, lr_0 = 6.2739e-04
Validation auc = 0.745070
Epoch 280
Validation auc = 0.745070
Epoch 281
Validation auc = 0.739437
Epoch 282
Validation auc = 0.732394
Epoch 283
Validation auc = 0.732394
Epoch 284
Loss = 2.1668e-01, PNorm = 44.6429, GNorm = 2.5599, lr_0 = 6.2203e-04
Validation auc = 0.722535
Epoch 285
Validation auc = 0.714085
Epoch 286
Validation auc = 0.708920
Epoch 287
Validation auc = 0.708920
Epoch 288
Validation auc = 0.710798
Epoch 289
Loss = 2.8424e-01, PNorm = 44.7194, GNorm = 1.3332, lr_0 = 6.1671e-04
Validation auc = 0.712207
Epoch 290
Validation auc = 0.715493
Epoch 291
Validation auc = 0.721127
Epoch 292
Validation auc = 0.728638
Epoch 293
Validation auc = 0.733803
Epoch 294
Loss = 2.8917e-01, PNorm = 44.7856, GNorm = 3.2349, lr_0 = 6.1144e-04
Validation auc = 0.733803
Epoch 295
Validation auc = 0.738028
Epoch 296
Validation auc = 0.739437
Epoch 297
Validation auc = 0.739437
Epoch 298
Validation auc = 0.748826
Epoch 299
Loss = 2.6445e-01, PNorm = 44.8719, GNorm = 2.5701, lr_0 = 6.0621e-04
Validation auc = 0.759155
Model 0 best validation auc = 0.763850 on epoch 150
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.913305
Ensemble test auc = 0.913305
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.523944
Epoch 1
Validation auc = 0.680751
Epoch 2
Validation auc = 0.669014
Epoch 3
Validation auc = 0.669484
Epoch 4
Loss = 1.3235e+00, PNorm = 40.6679, GNorm = 10.8348, lr_0 = 6.5000e-04
Validation auc = 0.676056
Epoch 5
Validation auc = 0.673709
Epoch 6
Validation auc = 0.669014
Epoch 7
Validation auc = 0.662441
Epoch 8
Validation auc = 0.657277
Epoch 9
Loss = 9.6073e-01, PNorm = 40.7089, GNorm = 10.5573, lr_0 = 9.9743e-04
Validation auc = 0.638967
Epoch 10
Validation auc = 0.656338
Epoch 11
Validation auc = 0.657277
Epoch 12
Validation auc = 0.656808
Epoch 13
Validation auc = 0.662911
Epoch 14
Loss = 7.9041e-01, PNorm = 40.7595, GNorm = 8.7374, lr_0 = 9.8890e-04
Validation auc = 0.654460
Epoch 15
Validation auc = 0.696714
Epoch 16
Validation auc = 0.658685
Epoch 17
Validation auc = 0.676526
Epoch 18
Validation auc = 0.673239
Epoch 19
Loss = 8.1752e-01, PNorm = 40.8060, GNorm = 8.7018, lr_0 = 9.8045e-04
Validation auc = 0.673239
Epoch 20
Validation auc = 0.690610
Epoch 21
Validation auc = 0.698122
Epoch 22
Validation auc = 0.694836
Epoch 23
Validation auc = 0.692488
Epoch 24
Loss = 5.8658e-01, PNorm = 40.8512, GNorm = 4.6600, lr_0 = 9.7207e-04
Validation auc = 0.696244
Epoch 25
Validation auc = 0.693897
Epoch 26
Validation auc = 0.702817
Epoch 27
Validation auc = 0.697653
Epoch 28
Validation auc = 0.662911
Epoch 29
Loss = 6.6766e-01, PNorm = 40.8789, GNorm = 6.1643, lr_0 = 9.6376e-04
Validation auc = 0.657277
Epoch 30
Validation auc = 0.660094
Epoch 31
Validation auc = 0.653521
Epoch 32
Validation auc = 0.643192
Epoch 33
Validation auc = 0.643662
Epoch 34
Loss = 6.8848e-01, PNorm = 40.9078, GNorm = 5.5601, lr_0 = 9.5552e-04
Validation auc = 0.646948
Epoch 35
Validation auc = 0.674648
Epoch 36
Validation auc = 0.667136
Epoch 37
Validation auc = 0.666667
Epoch 38
Validation auc = 0.664789
Epoch 39
Loss = 5.7763e-01, PNorm = 40.9449, GNorm = 3.8607, lr_0 = 9.4735e-04
Validation auc = 0.680751
Epoch 40
Validation auc = 0.672300
Epoch 41
Validation auc = 0.664319
Epoch 42
Validation auc = 0.663380
Epoch 43
Validation auc = 0.671831
Epoch 44
Loss = 4.8183e-01, PNorm = 40.9914, GNorm = 3.5697, lr_0 = 9.3925e-04
Validation auc = 0.708920
Epoch 45
Validation auc = 0.690141
Epoch 46
Validation auc = 0.697653
Epoch 47
Validation auc = 0.684038
Epoch 48
Validation auc = 0.668545
Epoch 49
Loss = 6.0177e-01, PNorm = 41.0231, GNorm = 3.2645, lr_0 = 9.3122e-04
Validation auc = 0.666667
Epoch 50
Validation auc = 0.664319
Epoch 51
Validation auc = 0.668075
Epoch 52
Validation auc = 0.665258
Epoch 53
Validation auc = 0.665728
Epoch 54
Loss = 5.6454e-01, PNorm = 41.0591, GNorm = 2.8316, lr_0 = 9.2326e-04
Validation auc = 0.663380
Epoch 55
Validation auc = 0.657746
Epoch 56
Validation auc = 0.651643
Epoch 57
Validation auc = 0.662911
Epoch 58
Validation auc = 0.661972
Epoch 59
Loss = 5.8134e-01, PNorm = 41.1075, GNorm = 3.9781, lr_0 = 9.1537e-04
Validation auc = 0.656808
Epoch 60
Validation auc = 0.647418
Epoch 61
Validation auc = 0.644131
Epoch 62
Validation auc = 0.649765
Epoch 63
Validation auc = 0.653052
Epoch 64
Loss = 5.2201e-01, PNorm = 41.1730, GNorm = 3.2161, lr_0 = 9.0754e-04
Validation auc = 0.653521
Epoch 65
Validation auc = 0.656808
Epoch 66
Validation auc = 0.655869
Epoch 67
Validation auc = 0.651643
Epoch 68
Validation auc = 0.646948
Epoch 69
Loss = 5.9220e-01, PNorm = 41.2163, GNorm = 3.6572, lr_0 = 8.9978e-04
Validation auc = 0.649296
Epoch 70
Validation auc = 0.638967
Epoch 71
Validation auc = 0.629108
Epoch 72
Validation auc = 0.632864
Epoch 73
Validation auc = 0.638028
Epoch 74
Loss = 5.5007e-01, PNorm = 41.2861, GNorm = 3.2879, lr_0 = 8.9209e-04
Validation auc = 0.640845
Epoch 75
Validation auc = 0.642254
Epoch 76
Validation auc = 0.640845
Epoch 77
Validation auc = 0.630047
Epoch 78
Validation auc = 0.626291
Epoch 79
Loss = 5.0204e-01, PNorm = 41.3389, GNorm = 3.6914, lr_0 = 8.8447e-04
Validation auc = 0.623474
Epoch 80
Validation auc = 0.619718
Epoch 81
Validation auc = 0.623474
Epoch 82
Validation auc = 0.629577
Epoch 83
Validation auc = 0.636620
Epoch 84
Loss = 4.2371e-01, PNorm = 41.4078, GNorm = 3.2763, lr_0 = 8.7691e-04
Validation auc = 0.642254
Epoch 85
Validation auc = 0.640376
Epoch 86
Validation auc = 0.649296
Epoch 87
Validation auc = 0.655399
Epoch 88
Validation auc = 0.659155
Epoch 89
Loss = 4.7340e-01, PNorm = 41.4651, GNorm = 2.1608, lr_0 = 8.6941e-04
Validation auc = 0.651643
Epoch 90
Validation auc = 0.646009
Epoch 91
Validation auc = 0.656338
Epoch 92
Validation auc = 0.660094
Epoch 93
Validation auc = 0.662911
Epoch 94
Loss = 4.8955e-01, PNorm = 41.5409, GNorm = 2.3912, lr_0 = 8.6198e-04
Validation auc = 0.644131
Epoch 95
Validation auc = 0.646009
Epoch 96
Validation auc = 0.643662
Epoch 97
Validation auc = 0.638498
Epoch 98
Validation auc = 0.638498
Epoch 99
Loss = 4.7300e-01, PNorm = 41.5982, GNorm = 2.8803, lr_0 = 8.5461e-04
Validation auc = 0.634742
Epoch 100
Validation auc = 0.650235
Epoch 101
Validation auc = 0.665258
Epoch 102
Validation auc = 0.668075
Epoch 103
Validation auc = 0.662911
Epoch 104
Loss = 4.9217e-01, PNorm = 41.6819, GNorm = 2.8168, lr_0 = 8.4730e-04
Validation auc = 0.661033
Epoch 105
Validation auc = 0.657277
Epoch 106
Validation auc = 0.643662
Epoch 107
Validation auc = 0.639437
Epoch 108
Validation auc = 0.644601
Epoch 109
Loss = 4.1850e-01, PNorm = 41.7752, GNorm = 2.7337, lr_0 = 8.4006e-04
Validation auc = 0.648357
Epoch 110
Validation auc = 0.657746
Epoch 111
Validation auc = 0.658685
Epoch 112
Validation auc = 0.659155
Epoch 113
Validation auc = 0.652582
Epoch 114
Loss = 4.3138e-01, PNorm = 41.8783, GNorm = 2.6740, lr_0 = 8.3288e-04
Validation auc = 0.652113
Epoch 115
Validation auc = 0.652113
Epoch 116
Validation auc = 0.651643
Epoch 117
Validation auc = 0.653521
Epoch 118
Validation auc = 0.662441
Epoch 119
Loss = 4.1232e-01, PNorm = 41.9849, GNorm = 3.0759, lr_0 = 8.2576e-04
Validation auc = 0.668545
Epoch 120
Validation auc = 0.676526
Epoch 121
Validation auc = 0.668075
Epoch 122
Validation auc = 0.670423
Epoch 123
Validation auc = 0.675117
Epoch 124
Loss = 4.1703e-01, PNorm = 42.0741, GNorm = 2.3914, lr_0 = 8.1870e-04
Validation auc = 0.678404
Epoch 125
Validation auc = 0.675587
Epoch 126
Validation auc = 0.673709
Epoch 127
Validation auc = 0.673709
Epoch 128
Validation auc = 0.684038
Epoch 129
Loss = 4.0811e-01, PNorm = 42.1763, GNorm = 2.8827, lr_0 = 8.1170e-04
Validation auc = 0.680282
Epoch 130
Validation auc = 0.679343
Epoch 131
Validation auc = 0.688263
Epoch 132
Validation auc = 0.690141
Epoch 133
Validation auc = 0.687793
Epoch 134
Loss = 4.5039e-01, PNorm = 42.2607, GNorm = 3.5740, lr_0 = 8.0476e-04
Validation auc = 0.692488
Epoch 135
Validation auc = 0.695775
Epoch 136
Validation auc = 0.694836
Epoch 137
Validation auc = 0.700469
Epoch 138
Validation auc = 0.700939
Epoch 139
Loss = 4.2703e-01, PNorm = 42.3466, GNorm = 2.6820, lr_0 = 7.9788e-04
Validation auc = 0.698592
Epoch 140
Validation auc = 0.694836
Epoch 141
Validation auc = 0.703286
Epoch 142
Validation auc = 0.698592
Epoch 143
Validation auc = 0.694836
Epoch 144
Loss = 4.6522e-01, PNorm = 42.4414, GNorm = 2.5665, lr_0 = 7.9106e-04
Validation auc = 0.699061
Epoch 145
Validation auc = 0.708920
Epoch 146
Validation auc = 0.707512
Epoch 147
Validation auc = 0.712207
Epoch 148
Validation auc = 0.714085
Epoch 149
Loss = 3.5129e-01, PNorm = 42.5515, GNorm = 2.9054, lr_0 = 7.8430e-04
Validation auc = 0.712676
Epoch 150
Validation auc = 0.711737
Epoch 151
Validation auc = 0.712207
Epoch 152
Validation auc = 0.721127
Epoch 153
Validation auc = 0.717840
Epoch 154
Loss = 3.5110e-01, PNorm = 42.6587, GNorm = 2.8926, lr_0 = 7.7759e-04
Validation auc = 0.715023
Epoch 155
Validation auc = 0.715493
Epoch 156
Validation auc = 0.723474
Epoch 157
Validation auc = 0.729577
Epoch 158
Validation auc = 0.725352
Epoch 159
Loss = 3.2225e-01, PNorm = 42.7444, GNorm = 2.4868, lr_0 = 7.7095e-04
Validation auc = 0.713615
Epoch 160
Validation auc = 0.708920
Epoch 161
Validation auc = 0.717840
Epoch 162
Validation auc = 0.719249
Epoch 163
Validation auc = 0.711737
Epoch 164
Loss = 5.0058e-01, PNorm = 42.8353, GNorm = 3.8402, lr_0 = 7.6436e-04
Validation auc = 0.707981
Epoch 165
Validation auc = 0.712207
Epoch 166
Validation auc = 0.713615
Epoch 167
Validation auc = 0.706573
Epoch 168
Validation auc = 0.710329
Epoch 169
Loss = 4.2041e-01, PNorm = 42.9160, GNorm = 2.5037, lr_0 = 7.5782e-04
Validation auc = 0.716432
Epoch 170
Validation auc = 0.716901
Epoch 171
Validation auc = 0.723944
Epoch 172
Validation auc = 0.735681
Epoch 173
Validation auc = 0.745540
Epoch 174
Loss = 3.7577e-01, PNorm = 43.0363, GNorm = 3.1675, lr_0 = 7.5134e-04
Validation auc = 0.748357
Epoch 175
Validation auc = 0.750704
Epoch 176
Validation auc = 0.739906
Epoch 177
Validation auc = 0.728638
Epoch 178
Validation auc = 0.730516
Epoch 179
Loss = 3.2812e-01, PNorm = 43.1468, GNorm = 2.8677, lr_0 = 7.4492e-04
Validation auc = 0.727700
Epoch 180
Validation auc = 0.745540
Epoch 181
Validation auc = 0.744601
Epoch 182
Validation auc = 0.746948
Epoch 183
Validation auc = 0.746479
Epoch 184
Loss = 3.6043e-01, PNorm = 43.2585, GNorm = 3.0121, lr_0 = 7.3855e-04
Validation auc = 0.738028
Epoch 185
Validation auc = 0.729108
Epoch 186
Validation auc = 0.725822
Epoch 187
Validation auc = 0.722066
Epoch 188
Validation auc = 0.714554
Epoch 189
Loss = 3.7800e-01, PNorm = 43.3535, GNorm = 2.7378, lr_0 = 7.3224e-04
Validation auc = 0.718779
Epoch 190
Validation auc = 0.722066
Epoch 191
Validation auc = 0.724883
Epoch 192
Validation auc = 0.722535
Epoch 193
Validation auc = 0.720657
Epoch 194
Loss = 3.5818e-01, PNorm = 43.4567, GNorm = 3.2224, lr_0 = 7.2598e-04
Validation auc = 0.717840
Epoch 195
Validation auc = 0.717371
Epoch 196
Validation auc = 0.718779
Epoch 197
Validation auc = 0.715962
Epoch 198
Validation auc = 0.717840
Epoch 199
Loss = 3.4056e-01, PNorm = 43.5553, GNorm = 2.2916, lr_0 = 7.1977e-04
Validation auc = 0.725822
Epoch 200
Validation auc = 0.733333
Epoch 201
Validation auc = 0.723944
Epoch 202
Validation auc = 0.723005
Epoch 203
Validation auc = 0.722535
Epoch 204
Loss = 3.3288e-01, PNorm = 43.6507, GNorm = 2.0318, lr_0 = 7.1362e-04
Validation auc = 0.722066
Epoch 205
Validation auc = 0.722535
Epoch 206
Validation auc = 0.725822
Epoch 207
Validation auc = 0.730986
Epoch 208
Validation auc = 0.740845
Epoch 209
Loss = 3.3900e-01, PNorm = 43.7688, GNorm = 2.3367, lr_0 = 7.0752e-04
Validation auc = 0.732864
Epoch 210
Validation auc = 0.723944
Epoch 211
Validation auc = 0.726291
Epoch 212
Validation auc = 0.730516
Epoch 213
Validation auc = 0.728169
Epoch 214
Loss = 3.6302e-01, PNorm = 43.8446, GNorm = 2.9859, lr_0 = 7.0147e-04
Validation auc = 0.748357
Epoch 215
Validation auc = 0.751643
Epoch 216
Validation auc = 0.755869
Epoch 217
Validation auc = 0.755399
Epoch 218
Validation auc = 0.749765
Epoch 219
Loss = 3.5437e-01, PNorm = 43.9305, GNorm = 2.6024, lr_0 = 6.9548e-04
Validation auc = 0.749765
Epoch 220
Validation auc = 0.745540
Epoch 221
Validation auc = 0.746948
Epoch 222
Validation auc = 0.747887
Epoch 223
Validation auc = 0.747418
Epoch 224
Loss = 2.8031e-01, PNorm = 44.0147, GNorm = 2.4544, lr_0 = 6.8953e-04
Validation auc = 0.749765
Epoch 225
Validation auc = 0.750235
Epoch 226
Validation auc = 0.757277
Epoch 227
Validation auc = 0.760094
Epoch 228
Validation auc = 0.768545
Epoch 229
Loss = 3.6740e-01, PNorm = 44.1040, GNorm = 2.4796, lr_0 = 6.8364e-04
Validation auc = 0.769484
Epoch 230
Validation auc = 0.770892
Epoch 231
Validation auc = 0.773709
Epoch 232
Validation auc = 0.765258
Epoch 233
Validation auc = 0.761972
Epoch 234
Loss = 2.5582e-01, PNorm = 44.1900, GNorm = 2.6549, lr_0 = 6.7779e-04
Validation auc = 0.756338
Epoch 235
Validation auc = 0.754930
Epoch 236
Validation auc = 0.752582
Epoch 237
Validation auc = 0.761033
Epoch 238
Validation auc = 0.761033
Epoch 239
Loss = 2.8032e-01, PNorm = 44.2854, GNorm = 2.3862, lr_0 = 6.7200e-04
Validation auc = 0.762441
Epoch 240
Validation auc = 0.761972
Epoch 241
Validation auc = 0.765258
Epoch 242
Validation auc = 0.756338
Epoch 243
Validation auc = 0.750704
Epoch 244
Loss = 2.8897e-01, PNorm = 44.3639, GNorm = 3.0296, lr_0 = 6.6625e-04
Validation auc = 0.737089
Epoch 245
Validation auc = 0.742723
Epoch 246
Validation auc = 0.746948
Epoch 247
Validation auc = 0.752113
Epoch 248
Validation auc = 0.749765
Epoch 249
Loss = 3.1562e-01, PNorm = 44.4432, GNorm = 1.9728, lr_0 = 6.6056e-04
Validation auc = 0.749765
Epoch 250
Validation auc = 0.753052
Epoch 251
Validation auc = 0.753052
Epoch 252
Validation auc = 0.746009
Epoch 253
Validation auc = 0.741784
Epoch 254
Loss = 3.7903e-01, PNorm = 44.5402, GNorm = 2.7568, lr_0 = 6.5491e-04
Validation auc = 0.737559
Epoch 255
Validation auc = 0.733333
Epoch 256
Validation auc = 0.734272
Epoch 257
Validation auc = 0.739906
Epoch 258
Validation auc = 0.749296
Epoch 259
Loss = 3.3796e-01, PNorm = 44.6455, GNorm = 1.8949, lr_0 = 6.4931e-04
Validation auc = 0.758685
Epoch 260
Validation auc = 0.754460
Epoch 261
Validation auc = 0.754460
Epoch 262
Validation auc = 0.750235
Epoch 263
Validation auc = 0.735211
Epoch 264
Loss = 2.6654e-01, PNorm = 44.7238, GNorm = 3.9596, lr_0 = 6.4376e-04
Validation auc = 0.725352
Epoch 265
Validation auc = 0.720188
Epoch 266
Validation auc = 0.719249
Epoch 267
Validation auc = 0.720657
Epoch 268
Validation auc = 0.725822
Epoch 269
Loss = 2.6060e-01, PNorm = 44.7848, GNorm = 1.9669, lr_0 = 6.3826e-04
Validation auc = 0.727230
Epoch 270
Validation auc = 0.728638
Epoch 271
Validation auc = 0.727700
Epoch 272
Validation auc = 0.730986
Epoch 273
Validation auc = 0.736620
Epoch 274
Loss = 2.0992e-01, PNorm = 44.8446, GNorm = 1.6835, lr_0 = 6.3280e-04
Validation auc = 0.746479
Epoch 275
Validation auc = 0.751643
Epoch 276
Validation auc = 0.752113
Epoch 277
Validation auc = 0.751174
Epoch 278
Validation auc = 0.753052
Epoch 279
Loss = 3.3659e-01, PNorm = 44.9187, GNorm = 1.9939, lr_0 = 6.2739e-04
Validation auc = 0.747418
Epoch 280
Validation auc = 0.742723
Epoch 281
Validation auc = 0.739437
Epoch 282
Validation auc = 0.737089
Epoch 283
Validation auc = 0.732394
Epoch 284
Loss = 3.1821e-01, PNorm = 44.9958, GNorm = 3.0341, lr_0 = 6.2203e-04
Validation auc = 0.727700
Epoch 285
Validation auc = 0.733333
Epoch 286
Validation auc = 0.726761
Epoch 287
Validation auc = 0.729108
Epoch 288
Validation auc = 0.731925
Epoch 289
Loss = 3.7322e-01, PNorm = 45.0684, GNorm = 2.0801, lr_0 = 6.1671e-04
Validation auc = 0.734742
Epoch 290
Validation auc = 0.736620
Epoch 291
Validation auc = 0.733333
Epoch 292
Validation auc = 0.731455
Epoch 293
Validation auc = 0.733803
Epoch 294
Loss = 2.2038e-01, PNorm = 45.1355, GNorm = 2.4326, lr_0 = 6.1144e-04
Validation auc = 0.736620
Epoch 295
Validation auc = 0.735681
Epoch 296
Validation auc = 0.732394
Epoch 297
Validation auc = 0.731925
Epoch 298
Validation auc = 0.733333
Epoch 299
Loss = 1.8042e-01, PNorm = 45.2102, GNorm = 2.2650, lr_0 = 6.0621e-04
Validation auc = 0.730047
Model 0 best validation auc = 0.773709 on epoch 231
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.927152
Ensemble test auc = 0.927152
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.564319
Epoch 1
Validation auc = 0.687793
Epoch 2
Validation auc = 0.679343
Epoch 3
Validation auc = 0.688263
Epoch 4
Loss = 1.4051e+00, PNorm = 40.6672, GNorm = 11.7362, lr_0 = 6.5000e-04
Validation auc = 0.707042
Epoch 5
Validation auc = 0.727230
Epoch 6
Validation auc = 0.702817
Epoch 7
Validation auc = 0.699531
Epoch 8
Validation auc = 0.690610
Epoch 9
Loss = 9.0930e-01, PNorm = 40.7167, GNorm = 8.7234, lr_0 = 9.9743e-04
Validation auc = 0.692958
Epoch 10
Validation auc = 0.653991
Epoch 11
Validation auc = 0.620188
Epoch 12
Validation auc = 0.623005
Epoch 13
Validation auc = 0.619718
Epoch 14
Loss = 8.4163e-01, PNorm = 40.7653, GNorm = 8.5683, lr_0 = 9.8890e-04
Validation auc = 0.683568
Epoch 15
Validation auc = 0.682160
Epoch 16
Validation auc = 0.720657
Epoch 17
Validation auc = 0.682629
Epoch 18
Validation auc = 0.700939
Epoch 19
Loss = 6.5092e-01, PNorm = 40.8110, GNorm = 4.1391, lr_0 = 9.8045e-04
Validation auc = 0.750235
Epoch 20
Validation auc = 0.746009
Epoch 21
Validation auc = 0.715493
Epoch 22
Validation auc = 0.694366
Epoch 23
Validation auc = 0.694836
Epoch 24
Loss = 7.0423e-01, PNorm = 40.8549, GNorm = 5.6547, lr_0 = 9.7207e-04
Validation auc = 0.710329
Epoch 25
Validation auc = 0.689671
Epoch 26
Validation auc = 0.665728
Epoch 27
Validation auc = 0.649765
Epoch 28
Validation auc = 0.646948
Epoch 29
Loss = 6.3085e-01, PNorm = 40.8841, GNorm = 4.9898, lr_0 = 9.6376e-04
Validation auc = 0.654460
Epoch 30
Validation auc = 0.662441
Epoch 31
Validation auc = 0.669953
Epoch 32
Validation auc = 0.658216
Epoch 33
Validation auc = 0.670423
Epoch 34
Loss = 6.9486e-01, PNorm = 40.9205, GNorm = 5.9975, lr_0 = 9.5552e-04
Validation auc = 0.672770
Epoch 35
Validation auc = 0.678404
Epoch 36
Validation auc = 0.697653
Epoch 37
Validation auc = 0.718779
Epoch 38
Validation auc = 0.688732
Epoch 39
Loss = 6.3885e-01, PNorm = 40.9647, GNorm = 4.9264, lr_0 = 9.4735e-04
Validation auc = 0.689202
Epoch 40
Validation auc = 0.726761
Epoch 41
Validation auc = 0.745070
Epoch 42
Validation auc = 0.736150
Epoch 43
Validation auc = 0.708920
Epoch 44
Loss = 4.8536e-01, PNorm = 41.0070, GNorm = 3.8567, lr_0 = 9.3925e-04
Validation auc = 0.700469
Epoch 45
Validation auc = 0.673709
Epoch 46
Validation auc = 0.660094
Epoch 47
Validation auc = 0.650704
Epoch 48
Validation auc = 0.646009
Epoch 49
Loss = 5.8854e-01, PNorm = 41.0510, GNorm = 5.6494, lr_0 = 9.3122e-04
Validation auc = 0.644131
Epoch 50
Validation auc = 0.646948
Epoch 51
Validation auc = 0.649765
Epoch 52
Validation auc = 0.635681
Epoch 53
Validation auc = 0.637559
Epoch 54
Loss = 6.0475e-01, PNorm = 41.1008, GNorm = 3.8984, lr_0 = 9.2326e-04
Validation auc = 0.648826
Epoch 55
Validation auc = 0.648826
Epoch 56
Validation auc = 0.646948
Epoch 57
Validation auc = 0.645540
Epoch 58
Validation auc = 0.647418
Epoch 59
Loss = 6.2247e-01, PNorm = 41.1582, GNorm = 4.4028, lr_0 = 9.1537e-04
Validation auc = 0.665258
Epoch 60
Validation auc = 0.674648
Epoch 61
Validation auc = 0.669953
Epoch 62
Validation auc = 0.664319
Epoch 63
Validation auc = 0.662441
Epoch 64
Loss = 5.2638e-01, PNorm = 41.1913, GNorm = 3.4416, lr_0 = 9.0754e-04
Validation auc = 0.669014
Epoch 65
Validation auc = 0.681221
Epoch 66
Validation auc = 0.665728
Epoch 67
Validation auc = 0.667606
Epoch 68
Validation auc = 0.669953
Epoch 69
Loss = 5.2159e-01, PNorm = 41.2450, GNorm = 3.2208, lr_0 = 8.9978e-04
Validation auc = 0.670892
Epoch 70
Validation auc = 0.668545
Epoch 71
Validation auc = 0.693897
Epoch 72
Validation auc = 0.674178
Epoch 73
Validation auc = 0.659155
Epoch 74
Loss = 5.5974e-01, PNorm = 41.2845, GNorm = 2.7567, lr_0 = 8.9209e-04
Validation auc = 0.647418
Epoch 75
Validation auc = 0.632394
Epoch 76
Validation auc = 0.635681
Epoch 77
Validation auc = 0.644131
Epoch 78
Validation auc = 0.653991
Epoch 79
Loss = 5.5882e-01, PNorm = 41.3513, GNorm = 3.4976, lr_0 = 8.8447e-04
Validation auc = 0.656808
Epoch 80
Validation auc = 0.654460
Epoch 81
Validation auc = 0.657277
Epoch 82
Validation auc = 0.656338
Epoch 83
Validation auc = 0.656808
Epoch 84
Loss = 4.9009e-01, PNorm = 41.4135, GNorm = 3.0505, lr_0 = 8.7691e-04
Validation auc = 0.653521
Epoch 85
Validation auc = 0.651174
Epoch 86
Validation auc = 0.656808
Epoch 87
Validation auc = 0.669953
Epoch 88
Validation auc = 0.681221
Epoch 89
Loss = 5.8163e-01, PNorm = 41.4696, GNorm = 2.6076, lr_0 = 8.6941e-04
Validation auc = 0.671362
Epoch 90
Validation auc = 0.671362
Epoch 91
Validation auc = 0.673709
Epoch 92
Validation auc = 0.667606
Epoch 93
Validation auc = 0.662441
Epoch 94
Loss = 6.2416e-01, PNorm = 41.5626, GNorm = 3.7212, lr_0 = 8.6198e-04
Validation auc = 0.657746
Epoch 95
Validation auc = 0.655399
Epoch 96
Validation auc = 0.650235
Epoch 97
Validation auc = 0.646479
Epoch 98
Validation auc = 0.650704
Epoch 99
Loss = 5.6097e-01, PNorm = 41.6217, GNorm = 3.2718, lr_0 = 8.5461e-04
Validation auc = 0.662441
Epoch 100
Validation auc = 0.678404
Epoch 101
Validation auc = 0.686854
Epoch 102
Validation auc = 0.705164
Epoch 103
Validation auc = 0.707512
Epoch 104
Loss = 4.3506e-01, PNorm = 41.7065, GNorm = 2.9852, lr_0 = 8.4730e-04
Validation auc = 0.709859
Epoch 105
Validation auc = 0.709859
Epoch 106
Validation auc = 0.712676
Epoch 107
Validation auc = 0.710798
Epoch 108
Validation auc = 0.722066
Epoch 109
Loss = 4.6150e-01, PNorm = 41.7788, GNorm = 2.9168, lr_0 = 8.4006e-04
Validation auc = 0.724413
Epoch 110
Validation auc = 0.724883
Epoch 111
Validation auc = 0.725822
Epoch 112
Validation auc = 0.717840
Epoch 113
Validation auc = 0.714085
Epoch 114
Loss = 5.3994e-01, PNorm = 41.8744, GNorm = 3.8980, lr_0 = 8.3288e-04
Validation auc = 0.714085
Epoch 115
Validation auc = 0.705634
Epoch 116
Validation auc = 0.713615
Epoch 117
Validation auc = 0.736620
Epoch 118
Validation auc = 0.756808
Epoch 119
Loss = 5.4242e-01, PNorm = 41.9537, GNorm = 2.9311, lr_0 = 8.2576e-04
Validation auc = 0.768545
Epoch 120
Validation auc = 0.764789
Epoch 121
Validation auc = 0.753521
Epoch 122
Validation auc = 0.745070
Epoch 123
Validation auc = 0.739437
Epoch 124
Loss = 4.1503e-01, PNorm = 42.0662, GNorm = 2.6455, lr_0 = 8.1870e-04
Validation auc = 0.732864
Epoch 125
Validation auc = 0.730516
Epoch 126
Validation auc = 0.731455
Epoch 127
Validation auc = 0.727700
Epoch 128
Validation auc = 0.730986
Epoch 129
Loss = 3.8873e-01, PNorm = 42.1833, GNorm = 2.6015, lr_0 = 8.1170e-04
Validation auc = 0.727700
Epoch 130
Validation auc = 0.708920
Epoch 131
Validation auc = 0.706103
Epoch 132
Validation auc = 0.703756
Epoch 133
Validation auc = 0.705634
Epoch 134
Loss = 3.4970e-01, PNorm = 42.2865, GNorm = 2.1979, lr_0 = 8.0476e-04
Validation auc = 0.707512
Epoch 135
Validation auc = 0.707512
Epoch 136
Validation auc = 0.699061
Epoch 137
Validation auc = 0.696244
Epoch 138
Validation auc = 0.694836
Epoch 139
Loss = 3.5926e-01, PNorm = 42.3642, GNorm = 2.6086, lr_0 = 7.9788e-04
Validation auc = 0.695775
Epoch 140
Validation auc = 0.700469
Epoch 141
Validation auc = 0.702817
Epoch 142
Validation auc = 0.707981
Epoch 143
Validation auc = 0.708920
Epoch 144
Loss = 3.5823e-01, PNorm = 42.4626, GNorm = 2.9428, lr_0 = 7.9106e-04
Validation auc = 0.715023
Epoch 145
Validation auc = 0.728169
Epoch 146
Validation auc = 0.735681
Epoch 147
Validation auc = 0.730047
Epoch 148
Validation auc = 0.735681
Epoch 149
Loss = 4.0792e-01, PNorm = 42.5467, GNorm = 3.5513, lr_0 = 7.8430e-04
Validation auc = 0.728169
Epoch 150
Validation auc = 0.714085
Epoch 151
Validation auc = 0.711268
Epoch 152
Validation auc = 0.700469
Epoch 153
Validation auc = 0.688263
Epoch 154
Loss = 4.4545e-01, PNorm = 42.6251, GNorm = 3.1155, lr_0 = 7.7759e-04
Validation auc = 0.681221
Epoch 155
Validation auc = 0.681221
Epoch 156
Validation auc = 0.676526
Epoch 157
Validation auc = 0.685446
Epoch 158
Validation auc = 0.697653
Epoch 159
Loss = 3.7058e-01, PNorm = 42.6939, GNorm = 2.3437, lr_0 = 7.7095e-04
Validation auc = 0.704695
Epoch 160
Validation auc = 0.710798
Epoch 161
Validation auc = 0.711268
Epoch 162
Validation auc = 0.709859
Epoch 163
Validation auc = 0.706573
Epoch 164
Loss = 3.3321e-01, PNorm = 42.7721, GNorm = 3.6732, lr_0 = 7.6436e-04
Validation auc = 0.709390
Epoch 165
Validation auc = 0.709390
Epoch 166
Validation auc = 0.707512
Epoch 167
Validation auc = 0.710329
Epoch 168
Validation auc = 0.718779
Epoch 169
Loss = 3.2255e-01, PNorm = 42.8506, GNorm = 2.5339, lr_0 = 7.5782e-04
Validation auc = 0.730516
Epoch 170
Validation auc = 0.737559
Epoch 171
Validation auc = 0.741315
Epoch 172
Validation auc = 0.743662
Epoch 173
Validation auc = 0.738498
Epoch 174
Loss = 3.3780e-01, PNorm = 42.9225, GNorm = 2.9563, lr_0 = 7.5134e-04
Validation auc = 0.731455
Epoch 175
Validation auc = 0.728638
Epoch 176
Validation auc = 0.728638
Epoch 177
Validation auc = 0.725352
Epoch 178
Validation auc = 0.732864
Epoch 179
Loss = 3.8685e-01, PNorm = 43.0155, GNorm = 2.7695, lr_0 = 7.4492e-04
Validation auc = 0.732394
Epoch 180
Validation auc = 0.735681
Epoch 181
Validation auc = 0.738498
Epoch 182
Validation auc = 0.745070
Epoch 183
Validation auc = 0.743662
Epoch 184
Loss = 3.6530e-01, PNorm = 43.1054, GNorm = 2.9411, lr_0 = 7.3855e-04
Validation auc = 0.738498
Epoch 185
Validation auc = 0.733803
Epoch 186
Validation auc = 0.729577
Epoch 187
Validation auc = 0.732394
Epoch 188
Validation auc = 0.731925
Epoch 189
Loss = 3.4936e-01, PNorm = 43.1918, GNorm = 2.5728, lr_0 = 7.3224e-04
Validation auc = 0.730986
Epoch 190
Validation auc = 0.731925
Epoch 191
Validation auc = 0.731925
Epoch 192
Validation auc = 0.731925
Epoch 193
Validation auc = 0.733803
Epoch 194
Loss = 3.4340e-01, PNorm = 43.2851, GNorm = 2.7547, lr_0 = 7.2598e-04
Validation auc = 0.735211
Epoch 195
Validation auc = 0.745540
Epoch 196
Validation auc = 0.744601
Epoch 197
Validation auc = 0.747887
Epoch 198
Validation auc = 0.741315
Epoch 199
Loss = 3.3705e-01, PNorm = 43.3819, GNorm = 2.7729, lr_0 = 7.1977e-04
Validation auc = 0.746009
Epoch 200
Validation auc = 0.744601
Epoch 201
Validation auc = 0.738028
Epoch 202
Validation auc = 0.735681
Epoch 203
Validation auc = 0.731455
Epoch 204
Loss = 2.8032e-01, PNorm = 43.4714, GNorm = 2.1922, lr_0 = 7.1362e-04
Validation auc = 0.727700
Epoch 205
Validation auc = 0.724883
Epoch 206
Validation auc = 0.727230
Epoch 207
Validation auc = 0.727230
Epoch 208
Validation auc = 0.725822
Epoch 209
Loss = 3.2756e-01, PNorm = 43.5704, GNorm = 1.8834, lr_0 = 7.0752e-04
Validation auc = 0.726761
Epoch 210
Validation auc = 0.722066
Epoch 211
Validation auc = 0.723474
Epoch 212
Validation auc = 0.717371
Epoch 213
Validation auc = 0.718779
Epoch 214
Loss = 3.5166e-01, PNorm = 43.6739, GNorm = 3.3102, lr_0 = 7.0147e-04
Validation auc = 0.723474
Epoch 215
Validation auc = 0.727230
Epoch 216
Validation auc = 0.728638
Epoch 217
Validation auc = 0.728638
Epoch 218
Validation auc = 0.741784
Epoch 219
Loss = 2.2628e-01, PNorm = 43.7585, GNorm = 1.9857, lr_0 = 6.9548e-04
Validation auc = 0.753991
Epoch 220
Validation auc = 0.755399
Epoch 221
Validation auc = 0.755869
Epoch 222
Validation auc = 0.754930
Epoch 223
Validation auc = 0.750704
Epoch 224
Loss = 2.4273e-01, PNorm = 43.8471, GNorm = 1.8774, lr_0 = 6.8953e-04
Validation auc = 0.747887
Epoch 225
Validation auc = 0.753052
Epoch 226
Validation auc = 0.756338
Epoch 227
Validation auc = 0.753521
Epoch 228
Validation auc = 0.748357
Epoch 229
Loss = 2.7182e-01, PNorm = 43.9272, GNorm = 2.7130, lr_0 = 6.8364e-04
Validation auc = 0.742723
Epoch 230
Validation auc = 0.737559
Epoch 231
Validation auc = 0.736150
Epoch 232
Validation auc = 0.734742
Epoch 233
Validation auc = 0.732864
Epoch 234
Loss = 2.5148e-01, PNorm = 44.0142, GNorm = 1.9918, lr_0 = 6.7779e-04
Validation auc = 0.729577
Epoch 235
Validation auc = 0.730516
Epoch 236
Validation auc = 0.734742
Epoch 237
Validation auc = 0.736150
Epoch 238
Validation auc = 0.739437
Epoch 239
Loss = 2.0977e-01, PNorm = 44.1101, GNorm = 2.8347, lr_0 = 6.7200e-04
Validation auc = 0.733803
Epoch 240
Validation auc = 0.733333
Epoch 241
Validation auc = 0.732394
Epoch 242
Validation auc = 0.723474
Epoch 243
Validation auc = 0.730986
Epoch 244
Loss = 3.7964e-01, PNorm = 44.1979, GNorm = 2.6817, lr_0 = 6.6625e-04
Validation auc = 0.735211
Epoch 245
Validation auc = 0.732394
Epoch 246
Validation auc = 0.730986
Epoch 247
Validation auc = 0.734742
Epoch 248
Validation auc = 0.731925
Epoch 249
Loss = 3.3532e-01, PNorm = 44.2639, GNorm = 2.1740, lr_0 = 6.6056e-04
Validation auc = 0.736620
Epoch 250
Validation auc = 0.739437
Epoch 251
Validation auc = 0.738498
Epoch 252
Validation auc = 0.742254
Epoch 253
Validation auc = 0.746009
Epoch 254
Loss = 2.6617e-01, PNorm = 44.3378, GNorm = 2.7253, lr_0 = 6.5491e-04
Validation auc = 0.748357
Epoch 255
Validation auc = 0.750235
Epoch 256
Validation auc = 0.753052
Epoch 257
Validation auc = 0.754460
Epoch 258
Validation auc = 0.750704
Epoch 259
Loss = 4.4055e-01, PNorm = 44.4208, GNorm = 4.6089, lr_0 = 6.4931e-04
Validation auc = 0.747887
Epoch 260
Validation auc = 0.743192
Epoch 261
Validation auc = 0.737089
Epoch 262
Validation auc = 0.729108
Epoch 263
Validation auc = 0.723005
Epoch 264
Loss = 3.5164e-01, PNorm = 44.4848, GNorm = 2.3530, lr_0 = 6.4376e-04
Validation auc = 0.723474
Epoch 265
Validation auc = 0.723005
Epoch 266
Validation auc = 0.728169
Epoch 267
Validation auc = 0.731925
Epoch 268
Validation auc = 0.739906
Epoch 269
Loss = 2.2422e-01, PNorm = 44.5657, GNorm = 1.8320, lr_0 = 6.3826e-04
Validation auc = 0.746479
Epoch 270
Validation auc = 0.750704
Epoch 271
Validation auc = 0.747887
Epoch 272
Validation auc = 0.748826
Epoch 273
Validation auc = 0.750704
Epoch 274
Loss = 2.1286e-01, PNorm = 44.6566, GNorm = 1.4141, lr_0 = 6.3280e-04
Validation auc = 0.754460
Epoch 275
Validation auc = 0.753521
Epoch 276
Validation auc = 0.745540
Epoch 277
Validation auc = 0.740376
Epoch 278
Validation auc = 0.741784
Epoch 279
Loss = 2.7187e-01, PNorm = 44.7222, GNorm = 1.8185, lr_0 = 6.2739e-04
Validation auc = 0.747887
Epoch 280
Validation auc = 0.753052
Epoch 281
Validation auc = 0.752113
Epoch 282
Validation auc = 0.753991
Epoch 283
Validation auc = 0.754930
Epoch 284
Loss = 2.7825e-01, PNorm = 44.7941, GNorm = 4.0537, lr_0 = 6.2203e-04
Validation auc = 0.758216
Epoch 285
Validation auc = 0.757746
Epoch 286
Validation auc = 0.757277
Epoch 287
Validation auc = 0.746479
Epoch 288
Validation auc = 0.749765
Epoch 289
Loss = 2.1320e-01, PNorm = 44.8638, GNorm = 2.3243, lr_0 = 6.1671e-04
Validation auc = 0.747887
Epoch 290
Validation auc = 0.748826
Epoch 291
Validation auc = 0.754930
Epoch 292
Validation auc = 0.758685
Epoch 293
Validation auc = 0.750235
Epoch 294
Loss = 2.4331e-01, PNorm = 44.9449, GNorm = 2.9290, lr_0 = 6.1144e-04
Validation auc = 0.746009
Epoch 295
Validation auc = 0.744601
Epoch 296
Validation auc = 0.746479
Epoch 297
Validation auc = 0.742723
Epoch 298
Validation auc = 0.739437
Epoch 299
Loss = 2.3498e-01, PNorm = 45.0247, GNorm = 2.3303, lr_0 = 6.0621e-04
Validation auc = 0.735211
Model 0 best validation auc = 0.768545 on epoch 119
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.913305
Ensemble test auc = 0.913305
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.30000000000000004, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=300, bias=False)
        (W_h): Linear(in_features=300, out_features=300, bias=False)
        (W_o): Linear(in_features=433, out_features=300, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.30000000000000004, inplace=False)
    (1): Linear(in_features=1500, out_features=300, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.30000000000000004, inplace=False)
    (4): Linear(in_features=300, out_features=300, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.30000000000000004, inplace=False)
    (7): Linear(in_features=300, out_features=1, bias=True)
  )
)
Number of parameters = 805,201
Moving model to cuda
Epoch 0
Validation auc = 0.681221
Epoch 1
Validation auc = 0.677465
Epoch 2
Validation auc = 0.669953
Epoch 3
Validation auc = 0.662911
Epoch 4
Loss = 1.1727e+00, PNorm = 40.6676, GNorm = 14.6704, lr_0 = 6.5000e-04
Validation auc = 0.638028
Epoch 5
Validation auc = 0.654930
Epoch 6
Validation auc = 0.653991
Epoch 7
Validation auc = 0.648357
Epoch 8
Validation auc = 0.649296
Epoch 9
Loss = 8.6141e-01, PNorm = 40.7109, GNorm = 9.1591, lr_0 = 9.9743e-04
Validation auc = 0.627230
Epoch 10
Validation auc = 0.619718
Epoch 11
Validation auc = 0.644601
Epoch 12
Validation auc = 0.622535
Epoch 13
Validation auc = 0.618310
Epoch 14
Loss = 7.2814e-01, PNorm = 40.7650, GNorm = 6.6048, lr_0 = 9.8890e-04
Validation auc = 0.599061
Epoch 15
Validation auc = 0.637089
Epoch 16
Validation auc = 0.651174
Epoch 17
Validation auc = 0.652582
Epoch 18
Validation auc = 0.724413
Epoch 19
Loss = 6.7291e-01, PNorm = 40.8132, GNorm = 6.3667, lr_0 = 9.8045e-04
Validation auc = 0.657746
Epoch 20
Validation auc = 0.662441
Epoch 21
Validation auc = 0.662911
Epoch 22
Validation auc = 0.681690
Epoch 23
Validation auc = 0.687324
Epoch 24
Loss = 7.9424e-01, PNorm = 40.8495, GNorm = 7.9910, lr_0 = 9.7207e-04
Validation auc = 0.672300
Epoch 25
Validation auc = 0.674178
Epoch 26
Validation auc = 0.658685
Epoch 27
Validation auc = 0.655399
Epoch 28
Validation auc = 0.656338
Epoch 29
Loss = 5.4907e-01, PNorm = 40.8834, GNorm = 4.3092, lr_0 = 9.6376e-04
Validation auc = 0.656808
Epoch 30
Validation auc = 0.641784
Epoch 31
Validation auc = 0.609859
Epoch 32
Validation auc = 0.600469
Epoch 33
Validation auc = 0.607981
Epoch 34
Loss = 7.4850e-01, PNorm = 40.9016, GNorm = 6.1751, lr_0 = 9.5552e-04
Validation auc = 0.623474
Epoch 35
Validation auc = 0.663850
Epoch 36
Validation auc = 0.698592
Epoch 37
Validation auc = 0.701878
Epoch 38
Validation auc = 0.703756
Epoch 39
Loss = 6.8735e-01, PNorm = 40.9312, GNorm = 3.6573, lr_0 = 9.4735e-04
Validation auc = 0.728638
Epoch 40
Validation auc = 0.763850
Epoch 41
Validation auc = 0.732394
Epoch 42
Validation auc = 0.700469
Epoch 43
Validation auc = 0.695775
Epoch 44
Loss = 6.6924e-01, PNorm = 40.9551, GNorm = 5.3660, lr_0 = 9.3925e-04
Validation auc = 0.681221
Epoch 45
Validation auc = 0.681690
Epoch 46
Validation auc = 0.678404
Epoch 47
Validation auc = 0.654930
Epoch 48
Validation auc = 0.659624
Epoch 49
Loss = 6.4416e-01, PNorm = 40.9989, GNorm = 4.5010, lr_0 = 9.3122e-04
Validation auc = 0.665258
Epoch 50
Validation auc = 0.669484
Epoch 51
Validation auc = 0.659155
Epoch 52
Validation auc = 0.645070
Epoch 53
Validation auc = 0.623944
Epoch 54
Loss = 5.9989e-01, PNorm = 41.0301, GNorm = 5.4409, lr_0 = 9.2326e-04
Validation auc = 0.617371
Epoch 55
Validation auc = 0.619718
Epoch 56
Validation auc = 0.618310
Epoch 57
Validation auc = 0.618310
Epoch 58
Validation auc = 0.631455
Epoch 59
Loss = 5.9493e-01, PNorm = 41.0679, GNorm = 4.3275, lr_0 = 9.1537e-04
Validation auc = 0.637089
Epoch 60
Validation auc = 0.631925
Epoch 61
Validation auc = 0.631455
Epoch 62
Validation auc = 0.641315
Epoch 63
Validation auc = 0.656338
Epoch 64
Loss = 6.0904e-01, PNorm = 41.1015, GNorm = 3.8266, lr_0 = 9.0754e-04
Validation auc = 0.650235
Epoch 65
Validation auc = 0.646009
Epoch 66
Validation auc = 0.636620
Epoch 67
Validation auc = 0.631925
Epoch 68
Validation auc = 0.638028
Epoch 69
Loss = 4.7100e-01, PNorm = 41.1589, GNorm = 3.7963, lr_0 = 8.9978e-04
Validation auc = 0.646479
Epoch 70
Validation auc = 0.649765
Epoch 71
Validation auc = 0.637559
Epoch 72
Validation auc = 0.633333
Epoch 73
Validation auc = 0.631455
Epoch 74
Loss = 5.6367e-01, PNorm = 41.2120, GNorm = 4.3209, lr_0 = 8.9209e-04
Validation auc = 0.630516
Epoch 75
Validation auc = 0.634742
Epoch 76
Validation auc = 0.641315
Epoch 77
Validation auc = 0.637089
Epoch 78
Validation auc = 0.634742
Epoch 79
Loss = 5.2709e-01, PNorm = 41.2384, GNorm = 3.4510, lr_0 = 8.8447e-04
Validation auc = 0.626761
Epoch 80
Validation auc = 0.624883
Epoch 81
Validation auc = 0.625822
Epoch 82
Validation auc = 0.630516
Epoch 83
Validation auc = 0.636620
Epoch 84
Loss = 4.4689e-01, PNorm = 41.3026, GNorm = 2.6538, lr_0 = 8.7691e-04
Validation auc = 0.635681
Epoch 85
Validation auc = 0.646479
Epoch 86
Validation auc = 0.648826
Epoch 87
Validation auc = 0.647418
Epoch 88
Validation auc = 0.653052
Epoch 89
Loss = 5.0990e-01, PNorm = 41.3538, GNorm = 3.5241, lr_0 = 8.6941e-04
Validation auc = 0.654930
Epoch 90
Validation auc = 0.658685
Epoch 91
Validation auc = 0.666197
Epoch 92
Validation auc = 0.665258
Epoch 93
Validation auc = 0.669953
Epoch 94
Loss = 4.5925e-01, PNorm = 41.4165, GNorm = 2.3186, lr_0 = 8.6198e-04
Validation auc = 0.666667
Epoch 95
Validation auc = 0.661033
Epoch 96
Validation auc = 0.647887
Epoch 97
Validation auc = 0.637559
Epoch 98
Validation auc = 0.630047
Epoch 99
Loss = 5.3186e-01, PNorm = 41.4778, GNorm = 3.0300, lr_0 = 8.5461e-04
Validation auc = 0.626761
Epoch 100
Validation auc = 0.625822
Epoch 101
Validation auc = 0.619718
Epoch 102
Validation auc = 0.628169
Epoch 103
Validation auc = 0.625822
Epoch 104
Loss = 5.0644e-01, PNorm = 41.5647, GNorm = 2.6269, lr_0 = 8.4730e-04
Validation auc = 0.626291
Epoch 105
Validation auc = 0.631925
Epoch 106
Validation auc = 0.637559
Epoch 107
Validation auc = 0.650235
Epoch 108
Validation auc = 0.660094
Epoch 109
Loss = 4.2352e-01, PNorm = 41.6538, GNorm = 2.4346, lr_0 = 8.4006e-04
Validation auc = 0.651643
Epoch 110
Validation auc = 0.643662
Epoch 111
Validation auc = 0.650235
Epoch 112
Validation auc = 0.652113
Epoch 113
Validation auc = 0.646009
Epoch 114
Loss = 5.1174e-01, PNorm = 41.7315, GNorm = 2.9920, lr_0 = 8.3288e-04
Validation auc = 0.652113
Epoch 115
Validation auc = 0.652113
Epoch 116
Validation auc = 0.646479
Epoch 117
Validation auc = 0.649765
Epoch 118
Validation auc = 0.650704
Epoch 119
Loss = 4.2323e-01, PNorm = 41.8124, GNorm = 2.3519, lr_0 = 8.2576e-04
Validation auc = 0.649296
Epoch 120
Validation auc = 0.647418
Epoch 121
Validation auc = 0.648826
Epoch 122
Validation auc = 0.643662
Epoch 123
Validation auc = 0.640376
Epoch 124
Loss = 4.2543e-01, PNorm = 41.8969, GNorm = 2.5333, lr_0 = 8.1870e-04
Validation auc = 0.644131
Epoch 125
Validation auc = 0.660094
Epoch 126
Validation auc = 0.658216
Epoch 127
Validation auc = 0.659624
Epoch 128
Validation auc = 0.659624
Epoch 129
Loss = 4.1787e-01, PNorm = 41.9872, GNorm = 1.8893, lr_0 = 8.1170e-04
Validation auc = 0.668075
Epoch 130
Validation auc = 0.680282
Epoch 131
Validation auc = 0.689671
Epoch 132
Validation auc = 0.694836
Epoch 133
Validation auc = 0.691549
Epoch 134
Loss = 5.1003e-01, PNorm = 42.0893, GNorm = 4.1963, lr_0 = 8.0476e-04
Validation auc = 0.676056
Epoch 135
Validation auc = 0.681221
Epoch 136
Validation auc = 0.684507
Epoch 137
Validation auc = 0.676526
Epoch 138
Validation auc = 0.679812
Epoch 139
Loss = 4.4795e-01, PNorm = 42.1844, GNorm = 2.8275, lr_0 = 7.9788e-04
Validation auc = 0.677465
Epoch 140
Validation auc = 0.690610
Epoch 141
Validation auc = 0.702347
Epoch 142
Validation auc = 0.684977
Epoch 143
Validation auc = 0.684507
Epoch 144
Loss = 4.3959e-01, PNorm = 42.2816, GNorm = 2.5273, lr_0 = 7.9106e-04
Validation auc = 0.687324
Epoch 145
Validation auc = 0.692019
Epoch 146
Validation auc = 0.710798
Epoch 147
Validation auc = 0.718779
Epoch 148
Validation auc = 0.721596
Epoch 149
Loss = 3.9867e-01, PNorm = 42.3744, GNorm = 2.1225, lr_0 = 7.8430e-04
Validation auc = 0.719718
Epoch 150
Validation auc = 0.723944
Epoch 151
Validation auc = 0.734272
Epoch 152
Validation auc = 0.740376
Epoch 153
Validation auc = 0.748826
Epoch 154
Loss = 5.2541e-01, PNorm = 42.4580, GNorm = 3.9638, lr_0 = 7.7759e-04
Validation auc = 0.757277
Epoch 155
Validation auc = 0.744601
Epoch 156
Validation auc = 0.729577
Epoch 157
Validation auc = 0.718779
Epoch 158
Validation auc = 0.713146
Epoch 159
Loss = 4.0985e-01, PNorm = 42.5545, GNorm = 2.7254, lr_0 = 7.7095e-04
Validation auc = 0.711737
Epoch 160
Validation auc = 0.718310
Epoch 161
Validation auc = 0.725822
Epoch 162
Validation auc = 0.731925
Epoch 163
Validation auc = 0.733333
Epoch 164
Loss = 4.2791e-01, PNorm = 42.6407, GNorm = 2.3567, lr_0 = 7.6436e-04
Validation auc = 0.730047
Epoch 165
Validation auc = 0.719718
Epoch 166
Validation auc = 0.715962
Epoch 167
Validation auc = 0.708920
Epoch 168
Validation auc = 0.703756
Epoch 169
Loss = 4.0718e-01, PNorm = 42.7321, GNorm = 2.4523, lr_0 = 7.5782e-04
Validation auc = 0.705164
Epoch 170
Validation auc = 0.700939
Epoch 171
Validation auc = 0.708920
Epoch 172
Validation auc = 0.715493
Epoch 173
Validation auc = 0.732394
Epoch 174
Loss = 4.5649e-01, PNorm = 42.8361, GNorm = 3.3361, lr_0 = 7.5134e-04
Validation auc = 0.737089
Epoch 175
Validation auc = 0.735211
Epoch 176
Validation auc = 0.730047
Epoch 177
Validation auc = 0.723944
Epoch 178
Validation auc = 0.707512
Epoch 179
Loss = 4.6537e-01, PNorm = 42.9396, GNorm = 3.4402, lr_0 = 7.4492e-04
Validation auc = 0.700469
Epoch 180
Validation auc = 0.704695
Epoch 181
Validation auc = 0.706573
Epoch 182
Validation auc = 0.722535
Epoch 183
Validation auc = 0.742723
Epoch 184
Loss = 4.4325e-01, PNorm = 43.0437, GNorm = 3.9401, lr_0 = 7.3855e-04
Validation auc = 0.746948
Epoch 185
Validation auc = 0.736620
Epoch 186
Validation auc = 0.716901
Epoch 187
Validation auc = 0.703286
Epoch 188
Validation auc = 0.693897
Epoch 189
Loss = 3.7391e-01, PNorm = 43.1419, GNorm = 2.0179, lr_0 = 7.3224e-04
Validation auc = 0.689671
Epoch 190
Validation auc = 0.697653
Epoch 191
Validation auc = 0.697653
Epoch 192
Validation auc = 0.702817
Epoch 193
Validation auc = 0.711268
Epoch 194
Loss = 3.6545e-01, PNorm = 43.2292, GNorm = 3.6288, lr_0 = 7.2598e-04
Validation auc = 0.723944
Epoch 195
Validation auc = 0.732864
Epoch 196
Validation auc = 0.732864
Epoch 197
Validation auc = 0.725822
Epoch 198
Validation auc = 0.725822
Epoch 199
Loss = 3.3024e-01, PNorm = 43.3228, GNorm = 2.1905, lr_0 = 7.1977e-04
Validation auc = 0.723005
Epoch 200
Validation auc = 0.724883
Epoch 201
Validation auc = 0.726291
Epoch 202
Validation auc = 0.727700
Epoch 203
Validation auc = 0.730047
Epoch 204
Loss = 4.2872e-01, PNorm = 43.4180, GNorm = 1.7103, lr_0 = 7.1362e-04
Validation auc = 0.731925
Epoch 205
Validation auc = 0.738028
Epoch 206
Validation auc = 0.738028
Epoch 207
Validation auc = 0.731925
Epoch 208
Validation auc = 0.725352
Epoch 209
Loss = 3.1306e-01, PNorm = 43.5044, GNorm = 2.8695, lr_0 = 7.0752e-04
Validation auc = 0.717840
Epoch 210
Validation auc = 0.711737
Epoch 211
Validation auc = 0.708920
Epoch 212
Validation auc = 0.713615
Epoch 213
Validation auc = 0.710329
Epoch 214
Loss = 2.8018e-01, PNorm = 43.5930, GNorm = 2.2115, lr_0 = 7.0147e-04
Validation auc = 0.704695
Epoch 215
Validation auc = 0.707512
Epoch 216
Validation auc = 0.710798
Epoch 217
Validation auc = 0.714554
Epoch 218
Validation auc = 0.715962
Epoch 219
Loss = 2.9224e-01, PNorm = 43.6914, GNorm = 2.0517, lr_0 = 6.9548e-04
Validation auc = 0.723474
Epoch 220
Validation auc = 0.723005
Epoch 221
Validation auc = 0.723005
Epoch 222
Validation auc = 0.719249
Epoch 223
Validation auc = 0.720188
Epoch 224
Loss = 3.1945e-01, PNorm = 43.7872, GNorm = 2.2772, lr_0 = 6.8953e-04
Validation auc = 0.723944
Epoch 225
Validation auc = 0.719249
Epoch 226
Validation auc = 0.716432
Epoch 227
Validation auc = 0.713146
Epoch 228
Validation auc = 0.708451
Epoch 229
Loss = 3.7921e-01, PNorm = 43.8907, GNorm = 3.8619, lr_0 = 6.8364e-04
Validation auc = 0.713146
Epoch 230
Validation auc = 0.720657
Epoch 231
Validation auc = 0.733333
Epoch 232
Validation auc = 0.748357
Epoch 233
Validation auc = 0.745070
Epoch 234
Loss = 2.9535e-01, PNorm = 43.9904, GNorm = 2.2146, lr_0 = 6.7779e-04
Validation auc = 0.729577
Epoch 235
Validation auc = 0.710798
Epoch 236
Validation auc = 0.710329
Epoch 237
Validation auc = 0.714085
Epoch 238
Validation auc = 0.710329
Epoch 239
Loss = 3.3344e-01, PNorm = 44.0809, GNorm = 3.4018, lr_0 = 6.7200e-04
Validation auc = 0.714554
Epoch 240
Validation auc = 0.722535
Epoch 241
Validation auc = 0.728169
Epoch 242
Validation auc = 0.738028
Epoch 243
Validation auc = 0.733333
Epoch 244
Loss = 2.3145e-01, PNorm = 44.1781, GNorm = 2.0004, lr_0 = 6.6625e-04
Validation auc = 0.724883
Epoch 245
Validation auc = 0.713615
Epoch 246
Validation auc = 0.713615
Epoch 247
Validation auc = 0.711737
Epoch 248
Validation auc = 0.712676
Epoch 249
Loss = 2.8079e-01, PNorm = 44.2733, GNorm = 2.0464, lr_0 = 6.6056e-04
Validation auc = 0.710798
Epoch 250
Validation auc = 0.715962
Epoch 251
Validation auc = 0.723474
Epoch 252
Validation auc = 0.732394
Epoch 253
Validation auc = 0.731925
Epoch 254
Loss = 2.6655e-01, PNorm = 44.3600, GNorm = 3.6381, lr_0 = 6.5491e-04
Validation auc = 0.727230
Epoch 255
Validation auc = 0.723474
Epoch 256
Validation auc = 0.722066
Epoch 257
Validation auc = 0.723005
Epoch 258
Validation auc = 0.728169
Epoch 259
Loss = 2.9938e-01, PNorm = 44.4516, GNorm = 3.5554, lr_0 = 6.4931e-04
Validation auc = 0.730047
Epoch 260
Validation auc = 0.732394
Epoch 261
Validation auc = 0.735211
Epoch 262
Validation auc = 0.741784
Epoch 263
Validation auc = 0.739906
Epoch 264
Loss = 3.0951e-01, PNorm = 44.5336, GNorm = 4.1220, lr_0 = 6.4376e-04
Validation auc = 0.734742
Epoch 265
Validation auc = 0.727700
Epoch 266
Validation auc = 0.721127
Epoch 267
Validation auc = 0.720188
Epoch 268
Validation auc = 0.721127
Epoch 269
Loss = 3.5341e-01, PNorm = 44.6227, GNorm = 3.4183, lr_0 = 6.3826e-04
Validation auc = 0.720657
Epoch 270
Validation auc = 0.720188
Epoch 271
Validation auc = 0.719718
Epoch 272
Validation auc = 0.726291
Epoch 273
Validation auc = 0.724413
Epoch 274
Loss = 3.0888e-01, PNorm = 44.7005, GNorm = 2.2725, lr_0 = 6.3280e-04
Validation auc = 0.727230
Epoch 275
Validation auc = 0.729577
Epoch 276
Validation auc = 0.731455
Epoch 277
Validation auc = 0.720657
Epoch 278
Validation auc = 0.723474
Epoch 279
Loss = 2.8115e-01, PNorm = 44.7759, GNorm = 2.6436, lr_0 = 6.2739e-04
Validation auc = 0.727700
Epoch 280
Validation auc = 0.725352
Epoch 281
Validation auc = 0.723005
Epoch 282
Validation auc = 0.722535
Epoch 283
Validation auc = 0.729577
Epoch 284
Loss = 2.7123e-01, PNorm = 44.8542, GNorm = 2.2509, lr_0 = 6.2203e-04
Validation auc = 0.730047
Epoch 285
Validation auc = 0.736620
Epoch 286
Validation auc = 0.742254
Epoch 287
Validation auc = 0.733803
Epoch 288
Validation auc = 0.727700
Epoch 289
Loss = 2.2580e-01, PNorm = 44.9367, GNorm = 1.7475, lr_0 = 6.1671e-04
Validation auc = 0.723474
Epoch 290
Validation auc = 0.717840
Epoch 291
Validation auc = 0.715493
Epoch 292
Validation auc = 0.711737
Epoch 293
Validation auc = 0.708920
Epoch 294
Loss = 2.3464e-01, PNorm = 45.0167, GNorm = 2.4002, lr_0 = 6.1144e-04
Validation auc = 0.716432
Epoch 295
Validation auc = 0.723944
Epoch 296
Validation auc = 0.730047
Epoch 297
Validation auc = 0.738028
Epoch 298
Validation auc = 0.737559
Epoch 299
Loss = 2.5384e-01, PNorm = 45.1094, GNorm = 1.6387, lr_0 = 6.0621e-04
Validation auc = 0.731455
Model 0 best validation auc = 0.763850 on epoch 40
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.852498
Ensemble test auc = 0.852498
10-fold cross validation
	Seed 0 ==> test auc = 0.856111
	Seed 1 ==> test auc = 0.779049
	Seed 2 ==> test auc = 0.921734
	Seed 3 ==> test auc = 0.797110
	Seed 4 ==> test auc = 0.918724
	Seed 5 ==> test auc = 0.919326
	Seed 6 ==> test auc = 0.913305
	Seed 7 ==> test auc = 0.927152
	Seed 8 ==> test auc = 0.913305
	Seed 9 ==> test auc = 0.852498
Overall test auc = 0.879831 +/- 0.052553
Elapsed time = 0:05:05
